-- Logs begin at Fri 2022-06-10 12:01:52 UTC, end at Fri 2022-06-10 13:20:43 UTC. -- Jun 10 12:43:25.830082 dv-cinder-master-3-smb systemd[1]: Starting Devstack devstack@keystone.service... Jun 10 12:43:25.841601 dv-cinder-master-3-smb devstack@keystone.service[80224]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 10 12:43:25.843717 dv-cinder-master-3-smb devstack@keystone.service[80224]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Jun 10 12:43:25 2022] *** Jun 10 12:43:25.843717 dv-cinder-master-3-smb devstack@keystone.service[80224]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 10 12:43:25.843717 dv-cinder-master-3-smb devstack@keystone.service[80224]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 10 12:43:25.843717 dv-cinder-master-3-smb devstack@keystone.service[80224]: nodename: dv-cinder-master-3-smb Jun 10 12:43:25.843717 dv-cinder-master-3-smb devstack@keystone.service[80224]: machine: x86_64 Jun 10 12:43:25.843717 dv-cinder-master-3-smb devstack@keystone.service[80224]: clock source: unix Jun 10 12:43:25.843717 dv-cinder-master-3-smb devstack@keystone.service[80224]: pcre jit disabled Jun 10 12:43:25.843717 dv-cinder-master-3-smb devstack@keystone.service[80224]: detected number of CPU cores: 4 Jun 10 12:43:25.843717 dv-cinder-master-3-smb devstack@keystone.service[80224]: current working directory: / Jun 10 12:43:25.843717 dv-cinder-master-3-smb devstack@keystone.service[80224]: detected binary path: /usr/bin/uwsgi-core Jun 10 12:43:25.843717 dv-cinder-master-3-smb devstack@keystone.service[80224]: your processes number limit is 47680 Jun 10 12:43:25.843717 dv-cinder-master-3-smb devstack@keystone.service[80224]: your memory page size is 4096 bytes Jun 10 12:43:25.843717 dv-cinder-master-3-smb devstack@keystone.service[80224]: detected max file descriptor number: 2048 Jun 10 12:43:25.843717 dv-cinder-master-3-smb devstack@keystone.service[80224]: lock engine: pthread robust mutexes Jun 10 12:43:25.844618 dv-cinder-master-3-smb devstack@keystone.service[80224]: thunder lock: enabled Jun 10 12:43:25.844618 dv-cinder-master-3-smb devstack@keystone.service[80224]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 10 12:43:25.844618 dv-cinder-master-3-smb devstack@keystone.service[80224]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 10 12:43:25.888354 dv-cinder-master-3-smb devstack@keystone.service[80224]: Python main interpreter initialized at 0x560d0d7829c0 Jun 10 12:43:25.888354 dv-cinder-master-3-smb devstack@keystone.service[80224]: python threads support enabled Jun 10 12:43:25.888354 dv-cinder-master-3-smb devstack@keystone.service[80224]: your server socket listen backlog is limited to 100 connections Jun 10 12:43:25.888354 dv-cinder-master-3-smb devstack@keystone.service[80224]: your mercy for graceful operations on workers is 90 seconds Jun 10 12:43:25.888702 dv-cinder-master-3-smb devstack@keystone.service[80224]: mapped 403077 bytes (393 KB) for 2 cores Jun 10 12:43:25.888804 dv-cinder-master-3-smb devstack@keystone.service[80224]: *** Operational MODE: preforking *** Jun 10 12:43:25.889086 dv-cinder-master-3-smb devstack@keystone.service[80224]: *** uWSGI is running in multiple interpreter mode *** Jun 10 12:43:25.889086 dv-cinder-master-3-smb devstack@keystone.service[80224]: spawned uWSGI master process (pid: 80224) Jun 10 12:43:25.889286 dv-cinder-master-3-smb systemd[1]: Started Devstack devstack@keystone.service. Jun 10 12:43:25.889842 dv-cinder-master-3-smb devstack@keystone.service[80224]: spawned uWSGI worker 1 (pid: 80237, cores: 1) Jun 10 12:43:25.890183 dv-cinder-master-3-smb devstack@keystone.service[80224]: spawned uWSGI worker 2 (pid: 80238, cores: 1) Jun 10 12:43:25.890340 dv-cinder-master-3-smb devstack@keystone.service[80224]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 10 12:43:31.438871 dv-cinder-master-3-smb devstack@keystone.service[80238]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 10 12:43:31.438871 dv-cinder-master-3-smb devstack@keystone.service[80238]: from cryptography.utils import int_from_bytes Jun 10 12:43:31.439659 dv-cinder-master-3-smb devstack@keystone.service[80238]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 10 12:43:31.439659 dv-cinder-master-3-smb devstack@keystone.service[80238]: from cryptography.utils import int_from_bytes Jun 10 12:43:31.442418 dv-cinder-master-3-smb devstack@keystone.service[80237]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 10 12:43:31.442418 dv-cinder-master-3-smb devstack@keystone.service[80237]: from cryptography.utils import int_from_bytes Jun 10 12:43:31.442899 dv-cinder-master-3-smb devstack@keystone.service[80237]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 10 12:43:31.442899 dv-cinder-master-3-smb devstack@keystone.service[80237]: from cryptography.utils import int_from_bytes Jun 10 12:43:31.492067 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 10 12:43:31.492268 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 10 12:43:31.492363 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 10 12:43:31.492458 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] command line args: [] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 10 12:43:31.492543 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 10 12:43:31.492624 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] command line args: [] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 10 12:43:31.492737 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 10 12:43:31.492894 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] ================================================================================ {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 10 12:43:31.492972 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 10 12:43:31.493056 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] ================================================================================ {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 10 12:43:31.493134 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] admin_token = **** {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:43:31.493310 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] admin_token = **** {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:43:31.493389 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] config_dir = [] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:43:31.493561 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] config_dir = [] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:43:31.493645 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:43:31.493725 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] config_source = [] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:43:31.493801 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:43:31.493908 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] debug = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:43:31.493985 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] config_source = [] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:43:31.494103 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] debug = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:43:31.494193 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:43:31.494414 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:43:31.494544 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:43:31.494640 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:43:31.494743 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:43:31.494858 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:43:31.494954 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:43:31.495053 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:43:31.495153 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:43:31.495253 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:43:31.495360 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] list_limit = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:43:31.495467 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] list_limit = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:43:31.495568 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] log_config_append = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:43:31.495669 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] log_config_append = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:43:31.495776 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:43:31.495916 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:43:31.498251 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] log_dir = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:43:31.498369 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] log_dir = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:43:31.498484 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] log_file = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:43:31.498553 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] log_file = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:43:31.498668 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:43:31.498749 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:43:31.498857 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:43:31.498934 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:43:31.499016 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:43:31.499092 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:43:31.499163 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:43:31.499289 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:43:31.499378 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:43:31.499458 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:43:31.499581 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:43:31.499663 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:43:31.499741 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:43:31.499888 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:43:31.499978 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:43:31.500052 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:43:31.500167 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:43:31.500241 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:43:31.500310 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:43:31.500379 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:43:31.500459 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:43:31.500536 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:43:31.500613 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:43:31.500689 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:43:31.500806 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:43:31.500893 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:43:31.500975 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:43:31.501096 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:43:31.501176 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:43:31.501254 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:43:31.501376 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.27/identity {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:43:31.501460 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.27/identity {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:43:31.501533 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] publish_errors = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:43:31.501655 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] publish_errors = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:43:31.501729 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:43:31.501805 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:43:31.501886 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:43:31.501957 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:43:31.502026 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:43:31.502102 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:43:31.502179 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:43:31.502269 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:43:31.502347 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:43:31.502423 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:43:31.502501 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] standard_threads = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:43:31.502578 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] standard_threads = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:43:31.502654 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:43:31.502730 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:43:31.502807 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:43:31.502807 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:43:31.502941 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:43:31.503008 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] use_journal = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:43:31.503082 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:43:31.503153 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] use_json = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:43:31.503222 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] use_journal = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:43:31.503291 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] use_stderr = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:43:31.503370 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] use_json = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:43:31.503441 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] use_syslog = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:43:31.503510 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] use_stderr = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:43:31.503510 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] use_syslog = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:43:31.503642 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:43:31.503711 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 10 12:43:31.503871 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.503949 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.504020 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.504090 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.504169 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.504272 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.504361 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.504515 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.504594 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.504768 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.504845 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.505012 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.505100 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.505252 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.505328 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.505549 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.505632 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.505805 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.505891 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.506058 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.506140 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.506299 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.506375 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.506535 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.506620 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.506782 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.506865 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.507019 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.507102 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.507268 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.507347 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.507505 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.507580 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.507741 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.507817 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.508029 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.508115 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.508279 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.508356 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.508519 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.508602 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.508756 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.508842 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.509014 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.509091 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.509265 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.509341 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.509517 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.509603 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.509751 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.509838 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.509984 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.510062 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.510217 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.510295 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.510453 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.510531 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.510704 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.510787 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.510953 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.511032 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.511186 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.511269 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.511420 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.511497 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.511683 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.511762 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.511955 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.512035 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.512201 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.512285 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.512441 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.512519 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.512676 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.512751 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.512915 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.512991 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.513151 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.513228 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.513384 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.513471 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.513617 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.513704 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.513847 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.513925 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.514080 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.514182 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.514313 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.514424 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.514552 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.514662 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.514783 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.514886 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.515023 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.515139 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.515247 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.515350 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.515481 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.515584 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.515716 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.515824 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.515969 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.516087 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.516212 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.516320 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.516522 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.516640 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.516736 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.516842 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.517080 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.517205 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.517205 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.517399 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.517495 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.517599 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.517701 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.517797 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.517898 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.517992 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.518105 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.518201 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.518336 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.518442 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.518573 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.518691 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.518810 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.518921 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.519060 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.519169 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.519296 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.519399 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.519530 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.519646 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.519770 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.519959 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.520072 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.520141 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.520311 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.520403 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.520604 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.522140 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.522424 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.522553 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.522689 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.522846 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.522947 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.523026 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.523201 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.523278 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.523454 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.523540 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.523706 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.523784 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.523980 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.524068 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.524228 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.524324 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.524473 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.524552 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.524720 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.524805 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.524964 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.525048 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.525208 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.525293 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.525450 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.525528 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.525696 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.525774 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.525952 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.526030 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.526200 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.526278 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.526444 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.526522 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.526687 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.526802 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.526931 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.527013 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.527188 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.527265 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.527439 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.527524 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.527677 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.527754 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.527934 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.528016 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.532613 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.532690 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.532769 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.532842 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.532914 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.532984 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.533061 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.533135 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.533206 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.533275 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.533275 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.533402 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.533402 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.533537 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.533537 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.533663 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.533733 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.533802 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.533802 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.533937 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.534007 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] auth.external = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.534083 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.534155 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.534224 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] auth.external = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.534224 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.534360 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.534360 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.534533 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.534607 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] auth.password = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.534678 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.534755 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] auth.token = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.534827 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] auth.password = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.534908 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.534987 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] auth.token = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.535059 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.535129 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.535206 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.535278 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.535356 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.535436 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.535514 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.535594 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.535667 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.535743 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.535822 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.535911 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.535994 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] credential.caching = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.536075 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.536153 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.536225 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] credential.caching = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.536304 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.536377 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.536448 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.536518 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.536589 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.536666 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.536739 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.536809 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.537138 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.537209 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.537280 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.537358 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.537431 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.537503 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.537573 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.537645 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.537715 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.537793 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.537867 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.537938 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.538008 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.538080 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.538150 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.538298 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.538380 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.538458 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.538531 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.538608 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.538680 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] federation.caching = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.538759 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.538833 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.538905 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] federation.caching = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.538976 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.539047 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.539118 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.539196 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.539270 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.539340 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.539411 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.539483 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.539554 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.539632 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.539706 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.539777 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.539844 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.539977 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.540040 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.540111 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.540174 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.540244 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.540315 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.540385 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.540483 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] identity.caching = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.540650 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.540766 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.540892 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] identity.caching = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.540979 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.541146 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.541225 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.541397 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.541478 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.541640 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.541719 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.541877 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.541967 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.542120 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.542201 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.542352 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.542433 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.542595 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.542674 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.543366 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.543451 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.543605 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.543684 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.543841 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.543938 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.544099 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.544180 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.544351 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.544429 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.544602 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.544681 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.544863 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.544963 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.545180 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.545285 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.545391 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.545494 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.545634 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.545719 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.545873 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.545952 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.546107 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.546194 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.546346 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.546425 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.546579 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.546658 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.546816 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.546903 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.547048 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.547127 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.547291 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.547378 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.547538 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.547631 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.547775 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.547855 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.548050 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.548140 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.548285 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.548371 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.548529 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.548607 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.548778 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.548921 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.549019 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.549111 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.549251 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.549330 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.549491 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.549569 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.549724 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.549811 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.549963 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.550043 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.550197 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.550277 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.550438 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.550524 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.550717 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.550832 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.551048 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.551142 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.551216 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.551296 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.551437 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.551516 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.551676 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.551754 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.551937 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.552034 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.552176 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.552255 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.552413 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.552493 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.552652 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.552738 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.552891 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.552971 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.553129 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.553207 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.553360 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.553446 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.553597 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.553676 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.553832 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.user = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.553912 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.554077 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.554165 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.554323 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.554401 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.554563 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.554641 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.554808 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.554899 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.555047 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.555126 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.555283 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.555371 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.555516 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.555604 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.555751 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.555830 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.556010 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.556089 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.556248 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.556339 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.556488 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.556566 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.556724 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.556804 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.556974 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.557061 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.557222 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.557305 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.557470 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.557548 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.557705 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.557792 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.557954 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.558033 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.558215 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.558302 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.558417 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.559150 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.559230 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.559303 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.559447 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.559527 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.559688 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.559774 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.559951 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.560036 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.560191 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.560269 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.561268 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.561379 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.561460 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.561533 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.561533 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.561662 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.561811 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.561888 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.561960 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.562032 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.562103 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.562174 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.562256 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.562330 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.562402 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.562473 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.562623 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.562764 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.562854 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.562930 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.563001 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] resource.caching = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.563072 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.563144 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.563221 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.563302 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.563375 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] resource.caching = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.563447 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.563519 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.563662 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.563742 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.563937 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.564020 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.564180 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.564258 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.564429 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.564507 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.564668 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.564749 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.564911 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.564990 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.565150 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] role.caching = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.565227 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.565385 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.565473 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] role.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.565622 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.565700 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] role.caching = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.565870 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] role.driver = sql {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.565948 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.566106 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.566187 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.566353 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.566430 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.566600 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.566688 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.566847 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.566928 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.567094 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.567171 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.567340 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.567418 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.567587 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.567666 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.567819 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.567946 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.568095 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.568176 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.568384 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.568463 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.568636 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.568713 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.568885 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.568963 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.569137 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.569303 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.569382 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.569544 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.569796 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.569893 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.570035 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.570115 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.570282 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.570360 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.570534 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.570619 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.570768 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.570846 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.571026 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.571105 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.571261 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.571346 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.571510 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.571588 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.571746 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.571824 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.572005 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.572098 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.572241 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.572320 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.572478 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.572556 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.572714 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.572802 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.572974 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.573073 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.573216 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.573335 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.573456 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.573590 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.573692 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.573831 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.573931 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] token.caching = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.574093 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.574198 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.574332 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.574452 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.574572 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] token.caching = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.574691 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.574814 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.574949 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.575066 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.575198 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.575304 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.575438 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.575564 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.575686 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.575815 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.575937 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.576084 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.576193 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.576333 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.576432 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.576578 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.576659 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.576830 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.576910 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.577072 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.577150 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.577314 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.577394 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.577554 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.577632 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.577806 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.577884 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.578059 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.578139 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.578294 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.578372 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] database.connection = **** {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.578540 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.578668 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.578833 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.578923 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.579076 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.579155 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] database.connection = **** {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.579365 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.579487 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.579610 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.579692 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.579852 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.579950 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.580112 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.580191 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.580352 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.580429 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.580580 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.580657 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.580820 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.580900 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.581063 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.581140 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.581302 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.581384 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.581539 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.581617 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.581774 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.581852 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.582013 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.582093 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.582243 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.582321 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.582487 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.582566 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.582726 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.582807 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.582898 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80237) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 10 12:43:31.582978 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.583188 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.583428 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 10 12:43:31.583600 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80238) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 10 12:43:31.699663 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 10 12:43:31.699830 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 10 12:43:31.700064 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 10 12:43:31.700180 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 10 12:43:31.700416 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 10 12:43:31.700643 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 10 12:43:31.709065 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 10 12:43:31.711850 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 10 12:43:31.737663 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 10 12:43:31.737894 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 10 12:43:31.738000 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 10 12:43:31.738160 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 10 12:43:31.798210 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 10 12:43:31.798701 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 10 12:43:31.850644 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 10 12:43:31.850876 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 10 12:43:31.850995 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 10 12:43:31.851231 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 10 12:43:31.916875 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 10 12:43:31.917098 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 10 12:43:31.917185 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 10 12:43:31.917313 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 10 12:43:31.917399 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 10 12:43:31.917533 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 10 12:43:31.917616 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 10 12:43:31.917748 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 10 12:43:31.917831 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 10 12:43:31.918506 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 10 12:43:31.918571 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 10 12:43:31.918779 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 10 12:43:31.941357 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 10 12:43:31.941495 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 10 12:43:31.941580 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 10 12:43:31.941726 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 10 12:43:31.941808 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 10 12:43:31.941947 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 10 12:43:31.942031 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 10 12:43:31.942165 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 10 12:43:31.942247 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 10 12:43:31.942378 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 10 12:43:31.942468 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 10 12:43:31.942588 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 10 12:43:31.942671 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 10 12:43:31.942799 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 10 12:43:31.953768 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 10 12:43:31.953869 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 10 12:43:31.966921 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 10 12:43:31.967144 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 10 12:43:31.967239 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 10 12:43:31.967364 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 10 12:43:31.967446 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 10 12:43:31.967584 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 10 12:43:31.967668 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 10 12:43:31.967800 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 10 12:43:31.967926 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 10 12:43:31.968036 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 10 12:43:31.968131 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 10 12:43:31.968248 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 10 12:43:31.968332 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80238) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 10 12:43:31.968461 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80237) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 10 12:43:31.985862 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:31.986076 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 10 12:43:31.986416 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:31.986500 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:31.986618 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 10 12:43:31.986700 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 10 12:43:31.986991 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:31.987081 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:31.987190 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 10 12:43:31.987314 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:31.987567 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:31.987646 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:31.987820 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:31.987937 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:31.988144 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:31.988399 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:31.998692 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:31.998970 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:31.999175 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:31.999259 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:31.999449 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:31.999531 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:31.999716 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.000023 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.007607 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80238) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 10 12:43:32.008126 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 10 12:43:32.011019 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80238) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 10 12:43:32.011300 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.011501 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 10 12:43:32.011596 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.011806 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.012010 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.012096 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.012171 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.012377 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.012463 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.012655 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.012737 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.012911 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.013001 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.013174 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.013256 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.013439 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.013519 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.013703 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.013792 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.013972 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.014238 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.041068 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.041395 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.042696 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80238) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 10 12:43:32.043538 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.043632 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 10 12:43:32.043823 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.048930 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80238) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 10 12:43:32.049129 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 10 12:43:32.049251 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.049403 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.049484 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.049678 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.056922 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80238) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 10 12:43:32.057104 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 10 12:43:32.057213 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.057384 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.061759 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80238) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 10 12:43:32.061900 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 10 12:43:32.062049 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.062184 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.062328 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.062461 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.062606 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.062739 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.062879 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.063010 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.063152 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.063281 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.063422 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.063548 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.063689 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.063812 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.082338 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.082430 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.082616 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.082700 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.087139 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80238) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 10 12:43:32.087224 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 10 12:43:32.091305 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.091477 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.091596 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.091680 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.096694 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 10 12:43:32.096798 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80238) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 10 12:43:32.100277 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 10 12:43:32.100376 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80238) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 10 12:43:32.103962 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.104048 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.104266 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.104346 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.104548 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.104632 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.104836 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.104919 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.105116 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.105206 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.105938 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.106033 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.124961 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 10 12:43:32.125055 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80238) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 10 12:43:32.125247 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.125330 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.125512 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.125594 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.125769 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.125851 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.133633 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.133718 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.135358 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.135458 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.135644 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.135726 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.138805 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80238) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 10 12:43:32.138896 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 10 12:43:32.139083 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.139178 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.140694 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.140779 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.140989 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.141071 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.141268 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.141352 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.155035 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 10 12:43:32.155130 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80238) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 10 12:43:32.155312 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.155406 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.155588 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.155670 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.155883 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.155966 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.156162 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.156256 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.156433 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.156516 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.156705 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.156796 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.175886 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 10 12:43:32.175993 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80238) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 10 12:43:32.179239 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 10 12:43:32.179335 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80238) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 10 12:43:32.182542 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.182642 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.184214 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.184338 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.186014 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 10 12:43:32.186148 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80238) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 10 12:43:32.186301 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.186437 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.186591 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.186744 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.194052 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.194216 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.195677 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 10 12:43:32.195825 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80238) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 10 12:43:32.198938 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.199153 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.199333 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.199436 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.199621 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.199705 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.199919 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.200016 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.209685 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80238) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 10 12:43:32.209916 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.210193 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.212312 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 10 12:43:32.212601 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.212876 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.218626 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80238) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 10 12:43:32.218905 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.219190 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.219460 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.219739 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.220035 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.220310 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.220582 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.220856 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.221230 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.221318 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80237) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 10 12:43:32.221500 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.221581 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.221769 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.221850 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.222033 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.222121 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.222292 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80238) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.222373 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.222604 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.222874 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.223138 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.223413 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.223789 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.224086 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.224364 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.224627 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.224941 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80237) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 10 12:43:32.260567 dv-cinder-master-3-smb devstack@keystone.service[80238]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x560d0d7829c0 pid: 80238 (default app) Jun 10 12:43:32.263623 dv-cinder-master-3-smb devstack@keystone.service[80237]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x560d0d7829c0 pid: 80237 (default app) Jun 10 12:43:32.264517 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23c53a1-bb76-4446-b293-d9df68d8a480 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:43:32.264815 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23c53a1-bb76-4446-b293-d9df68d8a480 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:43:32.265099 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23c53a1-bb76-4446-b293-d9df68d8a480 None None] PATH_INFO: `/v3` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:43:32.269313 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1/1] 10.222.0.27 () {50 vars in 791 bytes} [Fri Jun 10 12:43:32 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:43:40.640795 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd776c7-17d5-448c-8a79-7de6a0446845 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:43:40.641338 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd776c7-17d5-448c-8a79-7de6a0446845 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:43:40.641338 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd776c7-17d5-448c-8a79-7de6a0446845 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:43:40.646345 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1/2] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:43:40 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 10 12:43:40.651581 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d51f02d-c1ed-4139-8850-c97f0bf1ee9c None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:43:40.652044 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d51f02d-c1ed-4139-8850-c97f0bf1ee9c None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:43:40.652148 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d51f02d-c1ed-4139-8850-c97f0bf1ee9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:43:40.721387 dv-cinder-master-3-smb devstack@keystone.service[80238]: INFO dbcounter [None req-8d51f02d-c1ed-4139-8850-c97f0bf1ee9c None None] Registered counter for database keystone Jun 10 12:43:40.742619 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG oslo_db.sqlalchemy.engines [None req-8d51f02d-c1ed-4139-8850-c97f0bf1ee9c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80238) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 10 12:43:40.749339 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writer thread running {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 10 12:43:40.924403 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG passlib.handlers.bcrypt [None req-8d51f02d-c1ed-4139-8850-c97f0bf1ee9c None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80238) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 10 12:43:40.924756 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG passlib.handlers.bcrypt [None req-8d51f02d-c1ed-4139-8850-c97f0bf1ee9c None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80238) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 10 12:43:40.978152 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-8d51f02d-c1ed-4139-8850-c97f0bf1ee9c None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:43:40.979761 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-8d51f02d-c1ed-4139-8850-c97f0bf1ee9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:43:41.099385 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 2/3] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:43:40 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 449 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 10 12:43:41.240805 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd776c7-17d5-448c-8a79-7de6a0446845 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:43:41.310743 dv-cinder-master-3-smb devstack@keystone.service[80237]: INFO dbcounter [None req-7fd776c7-17d5-448c-8a79-7de6a0446845 None None] Registered counter for database keystone Jun 10 12:43:41.330033 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG oslo_db.sqlalchemy.engines [None req-7fd776c7-17d5-448c-8a79-7de6a0446845 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80237) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 10 12:43:41.336083 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writer thread running {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 10 12:43:41.453207 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd776c7-17d5-448c-8a79-7de6a0446845 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:43:41.457437 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03cdabf-aab8-45f8-9d00-7285323d12f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:43:41.458088 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03cdabf-aab8-45f8-9d00-7285323d12f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:43:41.458375 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03cdabf-aab8-45f8-9d00-7285323d12f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:43:41.458655 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03cdabf-aab8-45f8-9d00-7285323d12f1 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:43:41.467301 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03cdabf-aab8-45f8-9d00-7285323d12f1 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:43:41.519496 dv-cinder-master-3-smb devstack@keystone.service[80237]: WARNING keystone.common.rbac_enforcer.enforcer [None req-d03cdabf-aab8-45f8-9d00-7285323d12f1 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 10 12:43:41.519496 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03cdabf-aab8-45f8-9d00-7285323d12f1 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:43:41.529083 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 2/4] 10.222.0.27 () {58 vars in 1324 bytes} [Fri Jun 10 12:43:41 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 10 12:43:41.535076 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d51f02d-c1ed-4139-8850-c97f0bf1ee9c None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:43:41.552039 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d51f02d-c1ed-4139-8850-c97f0bf1ee9c None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:43:41.556420 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef989f7a-5cb7-4cdc-8bb7-c605791e6bb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:43:41.557526 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef989f7a-5cb7-4cdc-8bb7-c605791e6bb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:43:41.557818 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef989f7a-5cb7-4cdc-8bb7-c605791e6bb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:43:41.558099 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef989f7a-5cb7-4cdc-8bb7-c605791e6bb8 admin admin] PATH_INFO: `/v3/projects/2e3a9af772214953842e9cc63a52ae3a` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:43:41.559607 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef989f7a-5cb7-4cdc-8bb7-c605791e6bb8 admin admin] RBAC: Authorizing `identity:get_project(project_id=2e3a9af772214953842e9cc63a52ae3a)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:43:41.610261 dv-cinder-master-3-smb devstack@keystone.service[80238]: WARNING keystone.common.rbac_enforcer.enforcer [None req-ef989f7a-5cb7-4cdc-8bb7-c605791e6bb8 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 10 12:43:41.610608 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef989f7a-5cb7-4cdc-8bb7-c605791e6bb8 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:43:41.615247 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 3/5] 10.222.0.27 () {56 vars in 1207 bytes} [Fri Jun 10 12:43:41 2022] GET /identity/v3/projects/2e3a9af772214953842e9cc63a52ae3a => generated 344 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:43:43.686046 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8c02f7-c362-4700-8eb2-ee9678290585 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:43:43.686619 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8c02f7-c362-4700-8eb2-ee9678290585 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:43:43.686619 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8c02f7-c362-4700-8eb2-ee9678290585 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:43:43.690742 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 3/6] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:43:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:43:43.696059 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac03254a-11b3-4728-abdf-b2c639869bd9 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:43:43.696485 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac03254a-11b3-4728-abdf-b2c639869bd9 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:43:43.696707 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac03254a-11b3-4728-abdf-b2c639869bd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:43:43.716171 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-ac03254a-11b3-4728-abdf-b2c639869bd9 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:43:43.717722 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-ac03254a-11b3-4728-abdf-b2c639869bd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:43:43.725504 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 4/7] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:43:43 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 10 12:43:43.747389 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8c02f7-c362-4700-8eb2-ee9678290585 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:43:43.759188 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8c02f7-c362-4700-8eb2-ee9678290585 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:43:43.763949 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f153d6c-c8d4-4b63-826d-781e5d49d540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:43:43.764575 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f153d6c-c8d4-4b63-826d-781e5d49d540 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:43:43.764859 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f153d6c-c8d4-4b63-826d-781e5d49d540 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:43:43.765136 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f153d6c-c8d4-4b63-826d-781e5d49d540 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:43:43.772684 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f153d6c-c8d4-4b63-826d-781e5d49d540 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:43:43.774430 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f153d6c-c8d4-4b63-826d-781e5d49d540 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:43:43.783655 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 4/8] 10.222.0.27 () {58 vars in 1324 bytes} [Fri Jun 10 12:43:43 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 10 12:43:43.789073 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac03254a-11b3-4728-abdf-b2c639869bd9 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:43:43.797868 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac03254a-11b3-4728-abdf-b2c639869bd9 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:43:43.802199 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ba8171-e121-486b-9947-fe72393feb90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:43:43.803049 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ba8171-e121-486b-9947-fe72393feb90 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:43:43.803335 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ba8171-e121-486b-9947-fe72393feb90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:43:43.803614 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ba8171-e121-486b-9947-fe72393feb90 admin admin] PATH_INFO: `/v3/users/85098428641c41389cc157b4c057067c` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:43:43.805015 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ba8171-e121-486b-9947-fe72393feb90 admin admin] RBAC: Authorizing `identity:get_user(user_id=85098428641c41389cc157b4c057067c)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:43:43.806833 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ba8171-e121-486b-9947-fe72393feb90 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:43:43.808413 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 5/9] 10.222.0.27 () {56 vars in 1198 bytes} [Fri Jun 10 12:43:43 2022] GET /identity/v3/users/85098428641c41389cc157b4c057067c => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 12:43:45.954256 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b939ee3b-a61a-468a-a935-fbf5c399ea8f None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:43:45.954775 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b939ee3b-a61a-468a-a935-fbf5c399ea8f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:43:45.954877 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b939ee3b-a61a-468a-a935-fbf5c399ea8f None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:43:45.956178 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 5/10] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:43:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 10 12:43:45.961248 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd8a53f-1c67-4221-b09b-3b64029ddfaf None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:43:45.961652 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd8a53f-1c67-4221-b09b-3b64029ddfaf None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:43:45.961811 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd8a53f-1c67-4221-b09b-3b64029ddfaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:43:45.978695 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-1fd8a53f-1c67-4221-b09b-3b64029ddfaf None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:43:45.979922 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-1fd8a53f-1c67-4221-b09b-3b64029ddfaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:43:45.987317 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79616d1e-c281-42ea-a778-59bf0fe4cb58 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:43:45.987607 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79616d1e-c281-42ea-a778-59bf0fe4cb58 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:43:45.987912 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79616d1e-c281-42ea-a778-59bf0fe4cb58 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:43:45.989227 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 6/11] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:43:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 10 12:43:45.994302 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97129dd0-08c7-4b2a-81eb-a8704836d188 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:43:45.994589 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97129dd0-08c7-4b2a-81eb-a8704836d188 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:43:45.994870 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97129dd0-08c7-4b2a-81eb-a8704836d188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:43:46.034418 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG passlib.handlers.bcrypt [None req-97129dd0-08c7-4b2a-81eb-a8704836d188 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80237) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 10 12:43:46.034862 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG passlib.handlers.bcrypt [None req-97129dd0-08c7-4b2a-81eb-a8704836d188 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80237) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 10 12:43:46.061582 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 6/12] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:43:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 10 12:43:46.080557 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-97129dd0-08c7-4b2a-81eb-a8704836d188 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:43:46.082132 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-97129dd0-08c7-4b2a-81eb-a8704836d188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:43:46.084752 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd8a53f-1c67-4221-b09b-3b64029ddfaf None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:43:46.108511 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:43:46.180193 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 7/13] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:43:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 10 12:43:46.186526 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd8a53f-1c67-4221-b09b-3b64029ddfaf None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:43:46.202012 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97129dd0-08c7-4b2a-81eb-a8704836d188 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:43:46.220061 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878304be-6e0c-45a9-b81e-070d731332ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:43:46.220864 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878304be-6e0c-45a9-b81e-070d731332ce None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:43:46.221145 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878304be-6e0c-45a9-b81e-070d731332ce None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:43:46.221420 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878304be-6e0c-45a9-b81e-070d731332ce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:43:46.232470 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878304be-6e0c-45a9-b81e-070d731332ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:43:46.243485 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-878304be-6e0c-45a9-b81e-070d731332ce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 10 12:43:46.243485 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:43:46.243485 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:43:46.243485 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:43:46.243485 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:43:46.243485 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:43:46.243485 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:43:46.243485 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:43:46.243485 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:43:46.243485 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:43:46.243485 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:43:46.243485 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:43:46.243485 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:43:46.243485 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:43:46.243485 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:43:46.243485 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:43:46.244479 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:43:46.244479 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:43:46.244479 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:43:46.244479 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:43:46.244479 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:43:46.244479 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:43:46.244479 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:43:46.244479 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:43:46.244479 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:43:46.244479 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:43:46.244479 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:43:46.244479 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:43:46.244479 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:43:46.244479 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:43:46.244479 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:43:46.244479 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:43:46.245315 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:43:46.245315 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:43:46.245315 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:43:46.245315 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:43:46.245315 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:43:46.245315 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:43:46.245315 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:43:46.245315 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:43:46.245315 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:43:46.245315 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:43:46.245315 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 10 12:43:46.245315 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:43:46.245315 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 7/14] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Jun 10 12:43:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:43:46.251047 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878304be-6e0c-45a9-b81e-070d731332ce None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:43:46.306303 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97129dd0-08c7-4b2a-81eb-a8704836d188 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:43:46.309112 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:43:46.335615 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:43:46.344282 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26874ab1-7533-498b-a02e-497a972c82ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:43:46.345263 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26874ab1-7533-498b-a02e-497a972c82ad None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:43:46.345558 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26874ab1-7533-498b-a02e-497a972c82ad None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:43:46.345838 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26874ab1-7533-498b-a02e-497a972c82ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:43:46.353067 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878304be-6e0c-45a9-b81e-070d731332ce None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:43:46.387766 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d97a0e0-0c7f-4aa6-8af1-1203e3d1f4e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:43:46.388466 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d97a0e0-0c7f-4aa6-8af1-1203e3d1f4e1 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:43:46.388748 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d97a0e0-0c7f-4aa6-8af1-1203e3d1f4e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:43:46.389036 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d97a0e0-0c7f-4aa6-8af1-1203e3d1f4e1 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:43:46.389776 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d97a0e0-0c7f-4aa6-8af1-1203e3d1f4e1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:43:46.390081 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d97a0e0-0c7f-4aa6-8af1-1203e3d1f4e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:43:46.391471 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d97a0e0-0c7f-4aa6-8af1-1203e3d1f4e1 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:43:46.402920 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 8/15] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Jun 10 12:43:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 12:43:46.408297 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d97a0e0-0c7f-4aa6-8af1-1203e3d1f4e1 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:43:46.412481 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26874ab1-7533-498b-a02e-497a972c82ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:43:46.413988 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26874ab1-7533-498b-a02e-497a972c82ad None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:43:46.504816 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d97a0e0-0c7f-4aa6-8af1-1203e3d1f4e1 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:43:46.510487 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 8/16] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Jun 10 12:43:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 10 12:43:46.516021 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26874ab1-7533-498b-a02e-497a972c82ad None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:43:46.518667 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:43:46.539263 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b1fc46-3c19-47d2-a198-0865e871bb28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:43:46.540155 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b1fc46-3c19-47d2-a198-0865e871bb28 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:43:46.540441 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b1fc46-3c19-47d2-a198-0865e871bb28 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:43:46.540720 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b1fc46-3c19-47d2-a198-0865e871bb28 None admin] PATH_INFO: `/v3/users/85098428641c41389cc157b4c057067c` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:43:46.541924 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b1fc46-3c19-47d2-a198-0865e871bb28 None admin] RBAC: Authorizing `identity:get_user(user_id=85098428641c41389cc157b4c057067c)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:43:46.543912 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b1fc46-3c19-47d2-a198-0865e871bb28 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:43:46.545571 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 9/17] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Jun 10 12:43:46 2022] GET /identity/v3/users/85098428641c41389cc157b4c057067c => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 12:43:46.550807 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b1fc46-3c19-47d2-a198-0865e871bb28 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:43:46.565126 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:43:46.615375 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26874ab1-7533-498b-a02e-497a972c82ad None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:43:46.647092 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b1fc46-3c19-47d2-a198-0865e871bb28 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:43:46.649610 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d56f0a-50d8-4939-8cd0-b336aa530c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:43:46.650401 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d56f0a-50d8-4939-8cd0-b336aa530c7a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:43:46.650692 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d56f0a-50d8-4939-8cd0-b336aa530c7a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:43:46.650975 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d56f0a-50d8-4939-8cd0-b336aa530c7a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:43:46.669594 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d56f0a-50d8-4939-8cd0-b336aa530c7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:43:46.671051 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d56f0a-50d8-4939-8cd0-b336aa530c7a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:43:46.680567 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be022d7-ef7d-4324-a6ff-7f2fe75f9e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:43:46.681357 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be022d7-ef7d-4324-a6ff-7f2fe75f9e3e None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:43:46.681645 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be022d7-ef7d-4324-a6ff-7f2fe75f9e3e None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:43:46.681932 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be022d7-ef7d-4324-a6ff-7f2fe75f9e3e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:43:46.683685 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be022d7-ef7d-4324-a6ff-7f2fe75f9e3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:43:46.685110 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be022d7-ef7d-4324-a6ff-7f2fe75f9e3e None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:43:46.686708 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 10/18] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:43:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 12:43:46.691733 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-33d56f0a-50d8-4939-8cd0-b336aa530c7a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:43:46.691733 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:43:46.691733 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:43:46.691733 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:43:46.691733 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:43:46.691733 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:43:46.691733 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:43:46.691733 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:43:46.691733 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:43:46.691733 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:43:46.691733 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:43:46.691733 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:43:46.691733 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 10 12:43:46.691733 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 10 12:43:46.691733 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 10 12:43:46.692752 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 10 12:43:46.692752 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:43:46.692752 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:43:46.692752 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:43:46.692752 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:43:46.692752 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:43:46.692752 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:43:46.692752 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:43:46.692752 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:43:46.692752 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:43:46.692752 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:43:46.692752 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:43:46.692752 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:43:46.692752 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:43:46.692752 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:43:46.692752 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:43:46.692752 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:43:46.693631 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 10 12:43:46.693631 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 10 12:43:46.693631 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:43:46.693631 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:43:46.693877 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be022d7-ef7d-4324-a6ff-7f2fe75f9e3e None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:43:46.693949 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 9/19] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:43:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 10 12:43:46.698370 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d56f0a-50d8-4939-8cd0-b336aa530c7a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:43:46.754903 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:43:46.771831 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:43:46.790325 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be022d7-ef7d-4324-a6ff-7f2fe75f9e3e None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:43:46.798548 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d56f0a-50d8-4939-8cd0-b336aa530c7a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:43:46.825167 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4709ff-8149-4919-b411-dd5de086084f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:43:46.825819 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4709ff-8149-4919-b411-dd5de086084f None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:43:46.826104 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4709ff-8149-4919-b411-dd5de086084f None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:43:46.826383 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4709ff-8149-4919-b411-dd5de086084f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:43:46.827213 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4709ff-8149-4919-b411-dd5de086084f None admin] RBAC: Adding query filter params (role.id=b43d5a8247e244869c2c70f986f42e8a, scope.domain.id=default, user.id=85098428641c41389cc157b4c057067c) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:43:46.827525 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4709ff-8149-4919-b411-dd5de086084f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:43:46.829002 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4709ff-8149-4919-b411-dd5de086084f None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:43:46.831921 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeb4735-cc08-460f-8a9b-f20c5175a05e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:43:46.832571 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeb4735-cc08-460f-8a9b-f20c5175a05e None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:43:46.832855 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeb4735-cc08-460f-8a9b-f20c5175a05e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:43:46.833135 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeb4735-cc08-460f-8a9b-f20c5175a05e None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:43:46.833851 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eeb4735-cc08-460f-8a9b-f20c5175a05e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:43:46.834155 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eeb4735-cc08-460f-8a9b-f20c5175a05e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:43:46.835514 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eeb4735-cc08-460f-8a9b-f20c5175a05e None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:43:46.835882 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 11/20] 10.222.0.27 () {56 vars in 1322 bytes} [Fri Jun 10 12:43:46 2022] GET /identity/v3/role_assignments?user.id=85098428641c41389cc157b4c057067c&scope.domain.id=default&role.id=b43d5a8247e244869c2c70f986f42e8a => generated 233 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 12:43:46.855046 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 10/21] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Jun 10 12:43:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 12:43:49.398152 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f3f945-15d9-495d-b876-d8fb6955839d None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:43:49.398656 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f3f945-15d9-495d-b876-d8fb6955839d None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:43:49.398728 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f3f945-15d9-495d-b876-d8fb6955839d None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:43:49.400190 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 12/22] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:43:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:43:49.405801 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781f9839-bc25-4573-87b3-7f677ed82586 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:43:49.406250 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781f9839-bc25-4573-87b3-7f677ed82586 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:43:49.406353 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781f9839-bc25-4573-87b3-7f677ed82586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:43:49.432851 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-781f9839-bc25-4573-87b3-7f677ed82586 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:43:49.434906 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-781f9839-bc25-4573-87b3-7f677ed82586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:43:49.512292 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfde8815-114f-40c2-994f-add35598ee98 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:43:49.512936 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfde8815-114f-40c2-994f-add35598ee98 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:43:49.513430 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfde8815-114f-40c2-994f-add35598ee98 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:43:49.516479 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 13/23] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:43:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:43:49.517496 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 11/24] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:43:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 10 12:43:49.521057 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0322b1-3613-4bcd-b1a8-f53bdbaa6094 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:43:49.521576 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0322b1-3613-4bcd-b1a8-f53bdbaa6094 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:43:49.522088 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0322b1-3613-4bcd-b1a8-f53bdbaa6094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:43:49.554001 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-2e0322b1-3613-4bcd-b1a8-f53bdbaa6094 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:43:49.554421 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781f9839-bc25-4573-87b3-7f677ed82586 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:43:49.555543 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-2e0322b1-3613-4bcd-b1a8-f53bdbaa6094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:43:49.657128 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:43:49.685023 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 14/25] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:43:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 10 12:43:49.702053 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc29b729-dd5b-4d6b-8dcb-f6804dd2ada1 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:43:49.702053 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc29b729-dd5b-4d6b-8dcb-f6804dd2ada1 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:43:49.702053 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc29b729-dd5b-4d6b-8dcb-f6804dd2ada1 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:43:49.702053 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 15/26] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:43:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:43:49.702053 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131abcc7-4bbb-4d34-bae8-85f3fc203dff None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:43:49.702053 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131abcc7-4bbb-4d34-bae8-85f3fc203dff None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:43:49.702473 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131abcc7-4bbb-4d34-bae8-85f3fc203dff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:43:49.732538 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-131abcc7-4bbb-4d34-bae8-85f3fc203dff None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:43:49.733761 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-131abcc7-4bbb-4d34-bae8-85f3fc203dff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:43:49.777880 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781f9839-bc25-4573-87b3-7f677ed82586 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:43:49.845043 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 16/27] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:43:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 10 12:43:49.845928 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131abcc7-4bbb-4d34-bae8-85f3fc203dff None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:43:49.848296 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bb3e86-4d96-4fd4-8c81-27aa82aff0c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:43:49.849113 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bb3e86-4d96-4fd4-8c81-27aa82aff0c7 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:43:49.849405 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bb3e86-4d96-4fd4-8c81-27aa82aff0c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:43:49.849687 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bb3e86-4d96-4fd4-8c81-27aa82aff0c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:43:49.869113 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:43:49.880846 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bb3e86-4d96-4fd4-8c81-27aa82aff0c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:43:49.882361 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bb3e86-4d96-4fd4-8c81-27aa82aff0c7 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:43:49.910549 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-80bb3e86-4d96-4fd4-8c81-27aa82aff0c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:43:49.910549 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:43:49.910549 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:43:49.910549 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:43:49.910549 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:43:49.910549 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:43:49.910549 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:43:49.910549 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:43:49.910549 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:43:49.910549 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:43:49.910549 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:43:49.910549 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:43:49.910549 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 10 12:43:49.910549 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 10 12:43:49.910549 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 10 12:43:49.911404 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 10 12:43:49.911404 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:43:49.911404 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:43:49.911404 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:43:49.911404 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:43:49.911404 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:43:49.911404 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:43:49.911404 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:43:49.911404 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:43:49.911404 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:43:49.911404 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:43:49.911404 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:43:49.911404 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:43:49.911404 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:43:49.911404 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:43:49.911404 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:43:49.911404 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:43:49.912572 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 10 12:43:49.912572 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 10 12:43:49.912572 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:43:49.912572 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:43:49.921286 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 12/28] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:43:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 367 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:43:49.922125 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bb3e86-4d96-4fd4-8c81-27aa82aff0c7 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:43:49.998490 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131abcc7-4bbb-4d34-bae8-85f3fc203dff None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:43:50.038018 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1808810b-252a-4192-9dba-3fdf3bb572eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:43:50.039878 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1808810b-252a-4192-9dba-3fdf3bb572eb None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:43:50.040944 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1808810b-252a-4192-9dba-3fdf3bb572eb None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:43:50.041517 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1808810b-252a-4192-9dba-3fdf3bb572eb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:43:50.062083 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1808810b-252a-4192-9dba-3fdf3bb572eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:43:50.072622 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-1808810b-252a-4192-9dba-3fdf3bb572eb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 10 12:43:50.072622 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:43:50.072622 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:43:50.072622 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:43:50.072622 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:43:50.072622 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:43:50.072622 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:43:50.072622 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:43:50.072622 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:43:50.072622 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:43:50.072622 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:43:50.072622 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:43:50.072622 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:43:50.072622 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:43:50.072622 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:43:50.072622 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:43:50.073661 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:43:50.073661 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:43:50.073661 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:43:50.073661 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:43:50.073661 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:43:50.073661 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:43:50.073661 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:43:50.073661 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:43:50.073661 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:43:50.073661 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:43:50.073661 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:43:50.073661 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:43:50.073661 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:43:50.073661 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:43:50.073661 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:43:50.073661 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:43:50.074446 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:43:50.074446 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:43:50.074446 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:43:50.074446 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:43:50.074446 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:43:50.074446 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:43:50.074446 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:43:50.074446 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:43:50.074446 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:43:50.074446 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:43:50.074446 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 10 12:43:50.074446 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:43:50.076513 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 17/29] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Jun 10 12:43:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:43:50.077549 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1808810b-252a-4192-9dba-3fdf3bb572eb None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:43:50.103985 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:43:50.108106 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bb3e86-4d96-4fd4-8c81-27aa82aff0c7 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:43:50.183216 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cc5e7a-2b23-4ac2-bc6d-54cc575fd099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:43:50.184157 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc5e7a-2b23-4ac2-bc6d-54cc575fd099 None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:43:50.184613 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc5e7a-2b23-4ac2-bc6d-54cc575fd099 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:43:50.185044 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc5e7a-2b23-4ac2-bc6d-54cc575fd099 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:43:50.185958 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cc5e7a-2b23-4ac2-bc6d-54cc575fd099 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:43:50.187484 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cc5e7a-2b23-4ac2-bc6d-54cc575fd099 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:43:50.203019 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:43:50.242090 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1808810b-252a-4192-9dba-3fdf3bb572eb None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:43:50.277578 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eeb201-7ee8-4ddc-89fe-22e3be85ec63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:43:50.278498 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eeb201-7ee8-4ddc-89fe-22e3be85ec63 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:43:50.278924 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eeb201-7ee8-4ddc-89fe-22e3be85ec63 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:43:50.279338 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eeb201-7ee8-4ddc-89fe-22e3be85ec63 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:43:50.280263 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96eeb201-7ee8-4ddc-89fe-22e3be85ec63 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:43:50.280711 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96eeb201-7ee8-4ddc-89fe-22e3be85ec63 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:43:50.282243 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96eeb201-7ee8-4ddc-89fe-22e3be85ec63 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:43:50.307464 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 18/30] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Jun 10 12:43:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:43:50.308388 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eeb201-7ee8-4ddc-89fe-22e3be85ec63 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:43:50.404715 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 13/31] 10.222.0.27 () {60 vars in 1131 bytes} [Fri Jun 10 12:43:49 2022] POST /identity/v3/roles => generated 219 bytes in 483 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 12:43:50.406462 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a6f882-814d-4dc9-be59-51d48b491a60 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:43:50.407354 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a6f882-814d-4dc9-be59-51d48b491a60 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:43:50.407855 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a6f882-814d-4dc9-be59-51d48b491a60 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:43:50.409599 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 14/32] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:43:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:43:50.410602 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a6f882-814d-4dc9-be59-51d48b491a60 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:43:50.459954 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eeb201-7ee8-4ddc-89fe-22e3be85ec63 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:43:50.487178 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:43:50.498839 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd58457-d7a6-4ab6-9692-91de0b31dddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:43:50.499789 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd58457-d7a6-4ab6-9692-91de0b31dddf None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:43:50.500240 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd58457-d7a6-4ab6-9692-91de0b31dddf None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:43:50.508669 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd58457-d7a6-4ab6-9692-91de0b31dddf None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:43:50.508669 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd58457-d7a6-4ab6-9692-91de0b31dddf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:43:50.508669 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd58457-d7a6-4ab6-9692-91de0b31dddf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:43:50.508669 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd58457-d7a6-4ab6-9692-91de0b31dddf None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:43:50.514849 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 19/33] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Jun 10 12:43:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:43:50.516572 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43de3442-f44d-429f-b915-ad36551e77a2 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:43:50.517054 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43de3442-f44d-429f-b915-ad36551e77a2 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:43:50.517503 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43de3442-f44d-429f-b915-ad36551e77a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:43:50.539027 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-43de3442-f44d-429f-b915-ad36551e77a2 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:43:50.540676 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-43de3442-f44d-429f-b915-ad36551e77a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:43:50.564732 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a6f882-814d-4dc9-be59-51d48b491a60 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:43:50.625365 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e97a60c-cd95-4aad-aaed-dc7ef04fd515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:43:50.626048 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e97a60c-cd95-4aad-aaed-dc7ef04fd515 None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:43:50.626337 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e97a60c-cd95-4aad-aaed-dc7ef04fd515 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:43:50.626619 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e97a60c-cd95-4aad-aaed-dc7ef04fd515 None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:43:50.627599 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e97a60c-cd95-4aad-aaed-dc7ef04fd515 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:43:50.630464 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 20/34] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:43:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 10 12:43:50.631690 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e97a60c-cd95-4aad-aaed-dc7ef04fd515 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:43:50.632608 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4730ad-3a02-4888-b235-938ffc098182 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:43:50.634607 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4730ad-3a02-4888-b235-938ffc098182 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:43:50.635112 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4730ad-3a02-4888-b235-938ffc098182 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:43:50.638195 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 21/35] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:43:50 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:43:50.640399 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4730ad-3a02-4888-b235-938ffc098182 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:43:50.750877 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4730ad-3a02-4888-b235-938ffc098182 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:43:50.759733 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 15/36] 10.222.0.27 () {60 vars in 1141 bytes} [Fri Jun 10 12:43:50 2022] POST /identity/v3/projects => generated 301 bytes in 349 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 12:43:50.762008 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d05d75-6dba-477d-a8d3-f61df549da01 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:43:50.762367 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d05d75-6dba-477d-a8d3-f61df549da01 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:43:50.762709 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d05d75-6dba-477d-a8d3-f61df549da01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:43:50.777447 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:43:50.787812 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-a4d05d75-6dba-477d-a8d3-f61df549da01 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:43:50.789454 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a4d05d75-6dba-477d-a8d3-f61df549da01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:43:50.798096 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:43:50.806947 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32023b7-afc4-4c62-80ed-955a4c6f2af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:43:50.808699 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32023b7-afc4-4c62-80ed-955a4c6f2af0 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:43:50.809149 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32023b7-afc4-4c62-80ed-955a4c6f2af0 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:43:50.809566 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32023b7-afc4-4c62-80ed-955a4c6f2af0 None admin] PATH_INFO: `/v3/users/85098428641c41389cc157b4c057067c` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:43:50.811100 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32023b7-afc4-4c62-80ed-955a4c6f2af0 None admin] RBAC: Authorizing `identity:get_user(user_id=85098428641c41389cc157b4c057067c)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:43:50.812794 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32023b7-afc4-4c62-80ed-955a4c6f2af0 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:43:50.814771 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 22/37] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Jun 10 12:43:50 2022] GET /identity/v3/users/85098428641c41389cc157b4c057067c => generated 247 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:43:50.815881 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32023b7-afc4-4c62-80ed-955a4c6f2af0 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:43:50.883198 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 16/38] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:43:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 10 12:43:50.884461 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d05d75-6dba-477d-a8d3-f61df549da01 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:43:50.981131 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32023b7-afc4-4c62-80ed-955a4c6f2af0 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:43:51.029148 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d05d75-6dba-477d-a8d3-f61df549da01 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:43:51.032980 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e660f87f-4606-4e32-a32a-84f114da48c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:43:51.033853 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e660f87f-4606-4e32-a32a-84f114da48c3 None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:43:51.034136 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e660f87f-4606-4e32-a32a-84f114da48c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:43:51.034414 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e660f87f-4606-4e32-a32a-84f114da48c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:43:51.035227 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e660f87f-4606-4e32-a32a-84f114da48c3 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:43:51.036703 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e660f87f-4606-4e32-a32a-84f114da48c3 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:43:51.065215 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb209e2b-1bec-4ae1-a340-35b0faaa4999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:43:51.066019 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb209e2b-1bec-4ae1-a340-35b0faaa4999 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:43:51.066306 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb209e2b-1bec-4ae1-a340-35b0faaa4999 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:43:51.066589 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb209e2b-1bec-4ae1-a340-35b0faaa4999 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:43:51.067761 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb209e2b-1bec-4ae1-a340-35b0faaa4999 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:43:51.069233 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb209e2b-1bec-4ae1-a340-35b0faaa4999 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:43:51.070826 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 17/39] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:43:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:43:51.071598 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb209e2b-1bec-4ae1-a340-35b0faaa4999 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:43:51.081716 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:43:51.278415 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb209e2b-1bec-4ae1-a340-35b0faaa4999 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:43:51.311538 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd05665-9408-4ccf-8fcb-c4c3008bb4ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:43:51.312166 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd05665-9408-4ccf-8fcb-c4c3008bb4ca None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:43:51.312460 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd05665-9408-4ccf-8fcb-c4c3008bb4ca None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:43:51.312744 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd05665-9408-4ccf-8fcb-c4c3008bb4ca None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:43:51.313486 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd05665-9408-4ccf-8fcb-c4c3008bb4ca None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:43:51.314853 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd05665-9408-4ccf-8fcb-c4c3008bb4ca None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:43:51.321889 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 23/40] 10.222.0.27 () {60 vars in 1131 bytes} [Fri Jun 10 12:43:50 2022] POST /identity/v3/roles => generated 225 bytes in 506 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 12:43:51.323470 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc28964-f457-48e1-8372-d4e13b669613 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:43:51.323802 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc28964-f457-48e1-8372-d4e13b669613 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:43:51.324112 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc28964-f457-48e1-8372-d4e13b669613 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:43:51.325359 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 24/41] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:43:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:43:51.328115 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc28964-f457-48e1-8372-d4e13b669613 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:43:51.376483 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 18/42] 10.222.0.27 () {60 vars in 1131 bytes} [Fri Jun 10 12:43:51 2022] POST /identity/v3/roles => generated 223 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 12:43:51.377793 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78a661f-f0d5-4987-9c27-abbda7fb14f7 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:43:51.378089 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78a661f-f0d5-4987-9c27-abbda7fb14f7 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:43:51.378372 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78a661f-f0d5-4987-9c27-abbda7fb14f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:43:51.380237 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:43:51.399897 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-b78a661f-f0d5-4987-9c27-abbda7fb14f7 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:43:51.399897 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b78a661f-f0d5-4987-9c27-abbda7fb14f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:43:51.465115 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc28964-f457-48e1-8372-d4e13b669613 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:43:51.490422 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:43:51.497087 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 19/43] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:43:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 10 12:43:51.508526 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266cb49a-fe3f-4f39-b5b0-65b0bbfd7902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:43:51.509435 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266cb49a-fe3f-4f39-b5b0-65b0bbfd7902 None admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:43:51.509722 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266cb49a-fe3f-4f39-b5b0-65b0bbfd7902 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:43:51.510005 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266cb49a-fe3f-4f39-b5b0-65b0bbfd7902 None admin] PATH_INFO: `/v3/domains/default/users/85098428641c41389cc157b4c057067c/roles/b43d5a8247e244869c2c70f986f42e8a` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:43:51.512350 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266cb49a-fe3f-4f39-b5b0-65b0bbfd7902 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=85098428641c41389cc157b4c057067c, role_id=b43d5a8247e244869c2c70f986f42e8a)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:43:51.514024 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266cb49a-fe3f-4f39-b5b0-65b0bbfd7902 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:43:51.521114 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78a661f-f0d5-4987-9c27-abbda7fb14f7 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:43:51.564620 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 25/44] 10.222.0.27 () {58 vars in 1361 bytes} [Fri Jun 10 12:43:51 2022] PUT /identity/v3/domains/default/users/85098428641c41389cc157b4c057067c/roles/b43d5a8247e244869c2c70f986f42e8a => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 12:43:51.644523 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78a661f-f0d5-4987-9c27-abbda7fb14f7 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:43:51.682816 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eec6fa0-2cc1-41a5-87e1-0834c7afeccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:43:51.683670 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eec6fa0-2cc1-41a5-87e1-0834c7afeccb None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:43:51.683994 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eec6fa0-2cc1-41a5-87e1-0834c7afeccb None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:43:51.684282 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eec6fa0-2cc1-41a5-87e1-0834c7afeccb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:43:51.685535 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eec6fa0-2cc1-41a5-87e1-0834c7afeccb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:43:51.687017 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eec6fa0-2cc1-41a5-87e1-0834c7afeccb None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:43:51.688722 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 20/45] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:43:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 12:43:51.694475 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266cb49a-fe3f-4f39-b5b0-65b0bbfd7902 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:43:51.797410 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266cb49a-fe3f-4f39-b5b0-65b0bbfd7902 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:43:51.799684 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:43:51.830459 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e265c7f-a437-4291-b0da-b64b294ac4d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:43:51.831099 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e265c7f-a437-4291-b0da-b64b294ac4d9 None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:43:51.831644 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e265c7f-a437-4291-b0da-b64b294ac4d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:43:51.832178 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e265c7f-a437-4291-b0da-b64b294ac4d9 None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:43:51.833304 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e265c7f-a437-4291-b0da-b64b294ac4d9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:43:51.834758 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e265c7f-a437-4291-b0da-b64b294ac4d9 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:43:51.919282 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 26/46] 10.222.0.27 () {60 vars in 1141 bytes} [Fri Jun 10 12:43:51 2022] POST /identity/v3/projects => generated 312 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 12:43:53.678536 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863e1c2e-db84-48ae-a03c-9891d68b26e9 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:43:53.679076 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863e1c2e-db84-48ae-a03c-9891d68b26e9 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:43:53.679220 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863e1c2e-db84-48ae-a03c-9891d68b26e9 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:43:53.680514 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 21/47] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:43:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:43:53.685856 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7831677a-298a-4f81-8e44-9a3cd9dd0646 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:43:53.686255 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7831677a-298a-4f81-8e44-9a3cd9dd0646 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:43:53.686431 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7831677a-298a-4f81-8e44-9a3cd9dd0646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:43:53.703199 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-7831677a-298a-4f81-8e44-9a3cd9dd0646 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:43:53.704787 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-7831677a-298a-4f81-8e44-9a3cd9dd0646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:43:53.769145 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 27/48] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:43:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 10 12:43:53.791690 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863e1c2e-db84-48ae-a03c-9891d68b26e9 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:43:53.859794 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:43:53.893921 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863e1c2e-db84-48ae-a03c-9891d68b26e9 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:43:53.927749 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c30c53-fa7e-47dd-bbf4-aca7126a0544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:43:53.928621 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c30c53-fa7e-47dd-bbf4-aca7126a0544 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:43:53.928856 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c30c53-fa7e-47dd-bbf4-aca7126a0544 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:43:53.929136 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c30c53-fa7e-47dd-bbf4-aca7126a0544 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:43:53.940081 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c30c53-fa7e-47dd-bbf4-aca7126a0544 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:43:53.948766 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-17c30c53-fa7e-47dd-bbf4-aca7126a0544 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 10 12:43:53.948766 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:43:53.948766 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:43:53.948766 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:43:53.948766 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:43:53.948766 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:43:53.948766 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:43:53.948766 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:43:53.948766 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:43:53.948766 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:43:53.948766 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:43:53.948766 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:43:53.948766 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:43:53.948766 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:43:53.948766 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:43:53.948766 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:43:53.949631 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:43:53.949631 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:43:53.949631 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:43:53.949631 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:43:53.949631 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:43:53.949631 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:43:53.949631 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:43:53.949631 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:43:53.949631 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:43:53.949631 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:43:53.949631 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:43:53.949631 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:43:53.949631 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:43:53.949631 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:43:53.949631 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:43:53.949631 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:43:53.950406 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:43:53.950406 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:43:53.950406 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:43:53.950406 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:43:53.950406 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:43:53.950406 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:43:53.950406 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:43:53.950406 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:43:53.950406 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:43:53.950406 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:43:53.950406 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 10 12:43:53.950406 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:43:53.950406 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 22/49] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Jun 10 12:43:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:43:53.955683 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7831677a-298a-4f81-8e44-9a3cd9dd0646 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:43:54.042331 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:43:54.056731 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7831677a-298a-4f81-8e44-9a3cd9dd0646 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:43:54.074533 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba73fb0-a476-4fdf-b151-f596596afb4e None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:43:54.074828 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba73fb0-a476-4fdf-b151-f596596afb4e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:43:54.075112 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba73fb0-a476-4fdf-b151-f596596afb4e None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:43:54.076519 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 23/50] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:43:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 10 12:43:54.081267 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e476ac2-8c7f-47f6-b402-3d2fea3f593e None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:43:54.081620 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e476ac2-8c7f-47f6-b402-3d2fea3f593e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:43:54.091157 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e476ac2-8c7f-47f6-b402-3d2fea3f593e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:43:54.091932 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b46a3f7-5709-4d4c-b2ee-071f03c412ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:43:54.092608 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b46a3f7-5709-4d4c-b2ee-071f03c412ed None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:43:54.092893 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b46a3f7-5709-4d4c-b2ee-071f03c412ed None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:43:54.093172 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b46a3f7-5709-4d4c-b2ee-071f03c412ed None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:43:54.093926 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b46a3f7-5709-4d4c-b2ee-071f03c412ed None admin] RBAC: Adding query filter params (name=admin) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:43:54.094229 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b46a3f7-5709-4d4c-b2ee-071f03c412ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:43:54.095616 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b46a3f7-5709-4d4c-b2ee-071f03c412ed None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:43:54.098858 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-2e476ac2-8c7f-47f6-b402-3d2fea3f593e None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:43:54.100135 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2e476ac2-8c7f-47f6-b402-3d2fea3f593e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:43:54.104185 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 28/51] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Jun 10 12:43:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:43:54.109554 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b46a3f7-5709-4d4c-b2ee-071f03c412ed None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:43:54.166779 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 24/52] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:43:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 10 12:43:54.189900 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e476ac2-8c7f-47f6-b402-3d2fea3f593e None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:43:54.208332 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b46a3f7-5709-4d4c-b2ee-071f03c412ed None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:43:54.242356 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2730e1cb-2e7d-4aa0-b22a-0785afef8e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:43:54.243257 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2730e1cb-2e7d-4aa0-b22a-0785afef8e75 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:43:54.243547 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2730e1cb-2e7d-4aa0-b22a-0785afef8e75 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:43:54.243839 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2730e1cb-2e7d-4aa0-b22a-0785afef8e75 None admin] PATH_INFO: `/v3/users/85098428641c41389cc157b4c057067c` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:43:54.244070 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:43:54.245092 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2730e1cb-2e7d-4aa0-b22a-0785afef8e75 None admin] RBAC: Authorizing `identity:get_user(user_id=85098428641c41389cc157b4c057067c)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:43:54.246567 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2730e1cb-2e7d-4aa0-b22a-0785afef8e75 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:43:54.248234 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 29/53] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Jun 10 12:43:54 2022] GET /identity/v3/users/85098428641c41389cc157b4c057067c => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 12:43:54.253647 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2730e1cb-2e7d-4aa0-b22a-0785afef8e75 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:43:54.271838 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:43:54.289568 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e476ac2-8c7f-47f6-b402-3d2fea3f593e None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:43:54.327807 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce78b2c-625e-4e00-9cb5-b5775eab8dea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:43:54.328720 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce78b2c-625e-4e00-9cb5-b5775eab8dea None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:43:54.329091 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce78b2c-625e-4e00-9cb5-b5775eab8dea None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:43:54.329437 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce78b2c-625e-4e00-9cb5-b5775eab8dea None admin] PATH_INFO: `/v3/domains/default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:43:54.330685 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce78b2c-625e-4e00-9cb5-b5775eab8dea None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:43:54.332242 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce78b2c-625e-4e00-9cb5-b5775eab8dea None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:43:54.333851 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 25/54] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:43:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 12:43:54.339853 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce78b2c-625e-4e00-9cb5-b5775eab8dea None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:43:54.355789 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2730e1cb-2e7d-4aa0-b22a-0785afef8e75 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:43:54.388958 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8eb212-bafe-405a-9680-1329aa186beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:43:54.389748 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8eb212-bafe-405a-9680-1329aa186beb None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:43:54.390031 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8eb212-bafe-405a-9680-1329aa186beb None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:43:54.390308 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8eb212-bafe-405a-9680-1329aa186beb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:43:54.391477 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8eb212-bafe-405a-9680-1329aa186beb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:43:54.392950 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8eb212-bafe-405a-9680-1329aa186beb None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:43:54.394547 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 30/55] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:43:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 12:43:54.400158 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8eb212-bafe-405a-9680-1329aa186beb None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:43:54.438809 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce78b2c-625e-4e00-9cb5-b5775eab8dea None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:43:54.461498 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:43:54.472477 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22f7f2b-00c4-4b1a-9fed-7e8aa3484daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:43:54.473133 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22f7f2b-00c4-4b1a-9fed-7e8aa3484daa None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:43:54.473419 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22f7f2b-00c4-4b1a-9fed-7e8aa3484daa None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:43:54.473702 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22f7f2b-00c4-4b1a-9fed-7e8aa3484daa None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:43:54.474407 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22f7f2b-00c4-4b1a-9fed-7e8aa3484daa None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:43:54.475833 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22f7f2b-00c4-4b1a-9fed-7e8aa3484daa None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:43:54.482835 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:43:54.498945 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8eb212-bafe-405a-9680-1329aa186beb None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:43:54.531982 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22121e0-7ff4-4603-a140-277049c8e462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:43:54.532641 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22121e0-7ff4-4603-a140-277049c8e462 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:43:54.532925 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22121e0-7ff4-4603-a140-277049c8e462 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:43:54.533204 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22121e0-7ff4-4603-a140-277049c8e462 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:43:54.534080 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22121e0-7ff4-4603-a140-277049c8e462 None admin] RBAC: Adding query filter params (role.id=b43d5a8247e244869c2c70f986f42e8a, scope.domain.id=default, user.id=85098428641c41389cc157b4c057067c) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:43:54.534391 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22121e0-7ff4-4603-a140-277049c8e462 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:43:54.535842 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22121e0-7ff4-4603-a140-277049c8e462 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:43:54.541529 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 26/56] 10.222.0.27 () {60 vars in 1140 bytes} [Fri Jun 10 12:43:54 2022] POST /identity/v3/projects => generated 298 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 10 12:43:54.541916 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 31/57] 10.222.0.27 () {56 vars in 1322 bytes} [Fri Jun 10 12:43:54 2022] GET /identity/v3/role_assignments?user.id=85098428641c41389cc157b4c057067c&scope.domain.id=default&role.id=b43d5a8247e244869c2c70f986f42e8a => generated 530 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:43:56.656201 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18305b1-f04d-4ac1-a984-5045ac0103a8 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:43:56.656714 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18305b1-f04d-4ac1-a984-5045ac0103a8 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:43:56.656801 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18305b1-f04d-4ac1-a984-5045ac0103a8 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:43:56.658127 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 27/58] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:43:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:43:56.663641 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8083daef-2887-43b6-b4cf-68230bc84728 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:43:56.664081 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8083daef-2887-43b6-b4cf-68230bc84728 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:43:56.664241 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8083daef-2887-43b6-b4cf-68230bc84728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:43:56.681120 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-8083daef-2887-43b6-b4cf-68230bc84728 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:43:56.682298 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-8083daef-2887-43b6-b4cf-68230bc84728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:43:56.745618 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 32/59] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:43:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 10 12:43:56.768930 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18305b1-f04d-4ac1-a984-5045ac0103a8 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:43:56.867050 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18305b1-f04d-4ac1-a984-5045ac0103a8 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:43:56.900658 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48332e3-9d00-4548-ada7-3f43654dc78c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:43:56.901449 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48332e3-9d00-4548-ada7-3f43654dc78c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:43:56.901740 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48332e3-9d00-4548-ada7-3f43654dc78c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:43:56.902022 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48332e3-9d00-4548-ada7-3f43654dc78c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:43:56.903203 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48332e3-9d00-4548-ada7-3f43654dc78c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:43:56.904671 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48332e3-9d00-4548-ada7-3f43654dc78c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:43:56.906340 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 28/60] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:43:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 12:43:56.911679 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8083daef-2887-43b6-b4cf-68230bc84728 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:43:56.986983 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:43:57.007800 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8083daef-2887-43b6-b4cf-68230bc84728 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:43:57.040616 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c38c05d-e5c3-4330-9958-a1c7bcc62e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:43:57.041274 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c38c05d-e5c3-4330-9958-a1c7bcc62e99 None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:43:57.041560 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c38c05d-e5c3-4330-9958-a1c7bcc62e99 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:43:57.041845 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c38c05d-e5c3-4330-9958-a1c7bcc62e99 None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:43:57.043046 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c38c05d-e5c3-4330-9958-a1c7bcc62e99 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:43:57.044531 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c38c05d-e5c3-4330-9958-a1c7bcc62e99 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:43:57.134322 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 33/61] 10.222.0.27 () {60 vars in 1132 bytes} [Fri Jun 10 12:43:56 2022] POST /identity/v3/users => generated 275 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 12:44:00.072023 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17dab45-a122-47f1-9a1b-0fc588cd4574 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:00.073049 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17dab45-a122-47f1-9a1b-0fc588cd4574 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:00.074085 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17dab45-a122-47f1-9a1b-0fc588cd4574 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:00.075669 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 29/62] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:44:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:44:00.081180 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be76d767-e4d7-4578-aa34-d87e6c3011da None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:00.081990 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be76d767-e4d7-4578-aa34-d87e6c3011da None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:00.082471 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be76d767-e4d7-4578-aa34-d87e6c3011da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:00.103838 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-be76d767-e4d7-4578-aa34-d87e6c3011da None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:44:00.105417 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-be76d767-e4d7-4578-aa34-d87e6c3011da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:44:00.201238 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 34/63] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:44:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 10 12:44:00.237793 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17dab45-a122-47f1-9a1b-0fc588cd4574 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:00.288091 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:00.345406 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17dab45-a122-47f1-9a1b-0fc588cd4574 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:00.382394 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa3b315-66fd-44d0-98f2-cf5fc969bb0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:00.383216 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa3b315-66fd-44d0-98f2-cf5fc969bb0c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:00.383509 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa3b315-66fd-44d0-98f2-cf5fc969bb0c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:00.383797 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa3b315-66fd-44d0-98f2-cf5fc969bb0c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:00.395561 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa3b315-66fd-44d0-98f2-cf5fc969bb0c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:44:00.403352 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-baa3b315-66fd-44d0-98f2-cf5fc969bb0c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 10 12:44:00.403352 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:44:00.403352 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:44:00.403352 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:44:00.403352 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:44:00.403352 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:44:00.403352 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:44:00.403352 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:44:00.403352 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:44:00.403352 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:44:00.403352 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:44:00.403352 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:44:00.403352 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:44:00.403352 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:44:00.403352 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:44:00.403352 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:44:00.404214 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:44:00.404214 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:44:00.404214 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:44:00.404214 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:44:00.404214 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:44:00.404214 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:44:00.404214 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:44:00.404214 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:44:00.404214 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:44:00.404214 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:44:00.404214 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:44:00.404214 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:44:00.404214 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:44:00.404214 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:44:00.404214 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:44:00.404214 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:44:00.404983 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:44:00.404983 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:44:00.404983 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:44:00.404983 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:44:00.404983 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:44:00.404983 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:44:00.404983 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:44:00.404983 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:44:00.404983 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:44:00.404983 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:44:00.404983 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 10 12:44:00.404983 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:44:00.404983 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 30/64] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Jun 10 12:44:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:44:00.410412 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be76d767-e4d7-4578-aa34-d87e6c3011da None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:00.473255 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:00.520006 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be76d767-e4d7-4578-aa34-d87e6c3011da None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:00.570524 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a52ef04-e8de-41c5-a404-5b6bf5b30c85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:00.571234 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a52ef04-e8de-41c5-a404-5b6bf5b30c85 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:00.571530 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a52ef04-e8de-41c5-a404-5b6bf5b30c85 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:00.571819 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a52ef04-e8de-41c5-a404-5b6bf5b30c85 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:00.572649 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a52ef04-e8de-41c5-a404-5b6bf5b30c85 None admin] RBAC: Adding query filter params (name=member) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:00.572968 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a52ef04-e8de-41c5-a404-5b6bf5b30c85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:00.574538 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a52ef04-e8de-41c5-a404-5b6bf5b30c85 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:00.588484 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 35/65] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Jun 10 12:44:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:00.594858 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa3b315-66fd-44d0-98f2-cf5fc969bb0c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:00.669830 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544ec677-98a8-49f9-8a30-1b9f95cfb3f7 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:00.670415 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544ec677-98a8-49f9-8a30-1b9f95cfb3f7 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:00.670709 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544ec677-98a8-49f9-8a30-1b9f95cfb3f7 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:00.672322 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 36/66] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:44:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 10 12:44:00.678694 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f939fd-e5c7-4c0b-9602-f2c3abcc40f6 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:00.678998 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f939fd-e5c7-4c0b-9602-f2c3abcc40f6 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:00.679276 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f939fd-e5c7-4c0b-9602-f2c3abcc40f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:00.714847 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-b3f939fd-e5c7-4c0b-9602-f2c3abcc40f6 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:44:00.716468 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b3f939fd-e5c7-4c0b-9602-f2c3abcc40f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:44:00.728128 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa3b315-66fd-44d0-98f2-cf5fc969bb0c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:00.744296 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:00.766951 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9d4e02-aff1-481f-8f02-8ddbe2a04cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:00.767843 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9d4e02-aff1-481f-8f02-8ddbe2a04cc1 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:00.768954 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9d4e02-aff1-481f-8f02-8ddbe2a04cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:00.769056 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9d4e02-aff1-481f-8f02-8ddbe2a04cc1 None admin] PATH_INFO: `/v3/users/281b2f886b53468d808994f33b5f60ed` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:00.790279 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9d4e02-aff1-481f-8f02-8ddbe2a04cc1 None admin] RBAC: Authorizing `identity:get_user(user_id=281b2f886b53468d808994f33b5f60ed)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:00.792400 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9d4e02-aff1-481f-8f02-8ddbe2a04cc1 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:00.794906 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 31/67] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Jun 10 12:44:00 2022] GET /identity/v3/users/281b2f886b53468d808994f33b5f60ed => generated 275 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:00.796644 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0453ca67-40fd-4f79-8893-8f709102c8aa None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:00.797010 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0453ca67-40fd-4f79-8893-8f709102c8aa None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:00.797365 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0453ca67-40fd-4f79-8893-8f709102c8aa None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:00.798687 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 32/68] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:44:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:44:00.804686 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0453ca67-40fd-4f79-8893-8f709102c8aa None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:00.825252 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 37/69] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:44:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 10 12:44:00.833263 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297b9d8e-af88-4cc7-8a91-a1c2f5214617 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:00.833673 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297b9d8e-af88-4cc7-8a91-a1c2f5214617 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:00.834155 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297b9d8e-af88-4cc7-8a91-a1c2f5214617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:00.862824 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-297b9d8e-af88-4cc7-8a91-a1c2f5214617 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:44:00.864232 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-297b9d8e-af88-4cc7-8a91-a1c2f5214617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:44:00.924793 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0453ca67-40fd-4f79-8893-8f709102c8aa None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:00.965974 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185fc52e-2d67-4407-8bdd-6927933b5d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:00.966269 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:00.966642 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185fc52e-2d67-4407-8bdd-6927933b5d92 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:00.966935 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185fc52e-2d67-4407-8bdd-6927933b5d92 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:00.967226 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185fc52e-2d67-4407-8bdd-6927933b5d92 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:00.968246 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 38/70] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:44:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 10 12:44:00.977098 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297b9d8e-af88-4cc7-8a91-a1c2f5214617 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:01.017732 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:01.062237 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185fc52e-2d67-4407-8bdd-6927933b5d92 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:01.063708 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185fc52e-2d67-4407-8bdd-6927933b5d92 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:01.096432 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297b9d8e-af88-4cc7-8a91-a1c2f5214617 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:01.129691 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41208169-1451-4b2c-8f13-9d8d295fa10f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:01.130486 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41208169-1451-4b2c-8f13-9d8d295fa10f None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:01.130774 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41208169-1451-4b2c-8f13-9d8d295fa10f None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:01.131053 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41208169-1451-4b2c-8f13-9d8d295fa10f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:01.141459 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41208169-1451-4b2c-8f13-9d8d295fa10f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:44:01.149027 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-41208169-1451-4b2c-8f13-9d8d295fa10f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 10 12:44:01.149027 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:44:01.149027 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:44:01.149027 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:44:01.149027 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:44:01.149027 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:44:01.149027 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:44:01.149027 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:44:01.149027 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:44:01.149027 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:44:01.149027 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:44:01.149027 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:44:01.149027 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:44:01.149027 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:44:01.149027 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:44:01.149027 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:44:01.149908 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:44:01.149908 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:44:01.149908 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:44:01.149908 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:44:01.149908 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:44:01.149908 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:44:01.149908 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:44:01.149908 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:44:01.149908 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:44:01.149908 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:44:01.149908 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:44:01.149908 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:44:01.149908 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:44:01.149908 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:44:01.149908 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:44:01.149908 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:44:01.150704 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:44:01.150704 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:44:01.150704 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:44:01.150704 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:44:01.150704 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:44:01.150704 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:44:01.150704 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:44:01.150704 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:44:01.150704 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:44:01.150704 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:44:01.150704 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 10 12:44:01.150704 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:44:01.150704 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 39/71] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Jun 10 12:44:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:44:01.151347 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41208169-1451-4b2c-8f13-9d8d295fa10f None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:01.162990 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 33/72] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Jun 10 12:44:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 10 12:44:01.164397 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3946c158-5f03-4078-a33b-444051a45c2c None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:01.164695 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3946c158-5f03-4078-a33b-444051a45c2c None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:01.164984 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3946c158-5f03-4078-a33b-444051a45c2c None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:01.166522 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 34/73] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:44:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:44:01.167903 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa200d8-bfa6-4e1d-afe0-9e5ea7e1e9d5 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:01.168111 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa200d8-bfa6-4e1d-afe0-9e5ea7e1e9d5 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:01.168395 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa200d8-bfa6-4e1d-afe0-9e5ea7e1e9d5 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:01.169662 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 35/74] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:44:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:44:01.170388 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa200d8-bfa6-4e1d-afe0-9e5ea7e1e9d5 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:01.203225 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:01.231732 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:01.256151 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41208169-1451-4b2c-8f13-9d8d295fa10f None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:01.271736 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa200d8-bfa6-4e1d-afe0-9e5ea7e1e9d5 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:01.290368 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36163715-a244-407a-8c8a-3caeb2a365dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:01.291179 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36163715-a244-407a-8c8a-3caeb2a365dc None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:01.291471 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36163715-a244-407a-8c8a-3caeb2a365dc None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:01.291756 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36163715-a244-407a-8c8a-3caeb2a365dc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:01.301709 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36163715-a244-407a-8c8a-3caeb2a365dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:44:01.304816 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f86f0f-67e5-4dd7-afac-7ed18734a771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:01.305482 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f86f0f-67e5-4dd7-afac-7ed18734a771 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:01.305778 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f86f0f-67e5-4dd7-afac-7ed18734a771 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:01.306057 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f86f0f-67e5-4dd7-afac-7ed18734a771 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:01.306803 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f86f0f-67e5-4dd7-afac-7ed18734a771 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:01.307117 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f86f0f-67e5-4dd7-afac-7ed18734a771 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:01.309016 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f86f0f-67e5-4dd7-afac-7ed18734a771 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:01.310287 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-36163715-a244-407a-8c8a-3caeb2a365dc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 10 12:44:01.310287 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:44:01.310287 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:44:01.310287 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:44:01.310287 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:44:01.310287 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:44:01.310287 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:44:01.310287 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:44:01.310287 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:44:01.310287 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:44:01.310287 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:44:01.310287 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:44:01.310287 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:44:01.310287 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:44:01.310287 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:44:01.310287 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:44:01.311115 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:44:01.311115 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:44:01.311115 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:44:01.311115 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:44:01.311115 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:44:01.311115 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:44:01.311115 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:44:01.311115 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:44:01.311115 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:44:01.311115 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:44:01.311115 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:44:01.311115 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:44:01.311115 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:44:01.311115 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:44:01.311115 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:44:01.311115 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:44:01.311905 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:44:01.311905 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:44:01.311905 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:44:01.311905 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:44:01.311905 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:44:01.311905 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:44:01.311905 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:44:01.311905 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:44:01.311905 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:44:01.311905 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:44:01.311905 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 10 12:44:01.311905 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:44:01.311905 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 40/75] 10.222.0.27 () {56 vars in 1114 bytes} [Fri Jun 10 12:44:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:44:01.312554 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36163715-a244-407a-8c8a-3caeb2a365dc None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:01.324920 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 36/76] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Jun 10 12:44:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:01.326202 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbda6939-f7a5-479c-ae1b-e6f432db3cbb None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:01.326498 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbda6939-f7a5-479c-ae1b-e6f432db3cbb None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:01.326780 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbda6939-f7a5-479c-ae1b-e6f432db3cbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:01.342567 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-dbda6939-f7a5-479c-ae1b-e6f432db3cbb None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:44:01.343746 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-dbda6939-f7a5-479c-ae1b-e6f432db3cbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:44:01.408317 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 37/77] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:44:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 10 12:44:01.409639 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7539ad82-ee7a-4954-ac87-7599109330cf None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:01.409931 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7539ad82-ee7a-4954-ac87-7599109330cf None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:01.410211 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7539ad82-ee7a-4954-ac87-7599109330cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:01.410824 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36163715-a244-407a-8c8a-3caeb2a365dc None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:01.418606 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:01.426469 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-7539ad82-ee7a-4954-ac87-7599109330cf None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:44:01.427664 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-7539ad82-ee7a-4954-ac87-7599109330cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:44:01.446499 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2cb4b5-0522-4396-a37a-27a66f426eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:01.447362 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2cb4b5-0522-4396-a37a-27a66f426eb1 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:01.447645 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2cb4b5-0522-4396-a37a-27a66f426eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:01.447961 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2cb4b5-0522-4396-a37a-27a66f426eb1 None admin] PATH_INFO: `/v3/projects/3663edc029e8431d904a228a9e5755a0` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:01.449749 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2cb4b5-0522-4396-a37a-27a66f426eb1 None admin] RBAC: Authorizing `identity:get_project(project_id=3663edc029e8431d904a228a9e5755a0)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:01.451202 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2cb4b5-0522-4396-a37a-27a66f426eb1 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:01.453060 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 41/78] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Jun 10 12:44:01 2022] GET /identity/v3/projects/3663edc029e8431d904a228a9e5755a0 => generated 298 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:01.453659 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2cb4b5-0522-4396-a37a-27a66f426eb1 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:01.482609 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:01.500545 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 38/79] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:44:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 10 12:44:01.504868 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7539ad82-ee7a-4954-ac87-7599109330cf None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:01.554402 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2cb4b5-0522-4396-a37a-27a66f426eb1 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:01.587961 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dca7576-9820-4e0e-ac21-33a2be405373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:01.588627 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dca7576-9820-4e0e-ac21-33a2be405373 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:01.588942 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dca7576-9820-4e0e-ac21-33a2be405373 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:01.589224 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dca7576-9820-4e0e-ac21-33a2be405373 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:01.590012 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dca7576-9820-4e0e-ac21-33a2be405373 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:01.590322 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dca7576-9820-4e0e-ac21-33a2be405373 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:01.591722 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dca7576-9820-4e0e-ac21-33a2be405373 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:01.599458 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 42/80] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Jun 10 12:44:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:01.604169 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7539ad82-ee7a-4954-ac87-7599109330cf None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:01.605170 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dca7576-9820-4e0e-ac21-33a2be405373 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:01.638191 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71af237-13c1-47a4-b045-1ac78419f6d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:01.638653 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:01.639073 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71af237-13c1-47a4-b045-1ac78419f6d9 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:01.639361 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71af237-13c1-47a4-b045-1ac78419f6d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:01.639643 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71af237-13c1-47a4-b045-1ac78419f6d9 None admin] PATH_INFO: `/v3/users/85098428641c41389cc157b4c057067c` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:01.640938 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71af237-13c1-47a4-b045-1ac78419f6d9 None admin] RBAC: Authorizing `identity:get_user(user_id=85098428641c41389cc157b4c057067c)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:01.642437 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71af237-13c1-47a4-b045-1ac78419f6d9 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:01.644072 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 39/81] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Jun 10 12:44:01 2022] GET /identity/v3/users/85098428641c41389cc157b4c057067c => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:01.648945 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71af237-13c1-47a4-b045-1ac78419f6d9 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:01.698646 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:01.703109 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dca7576-9820-4e0e-ac21-33a2be405373 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:01.736050 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f62f8db-32f0-4801-a081-9b07dab6913b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:01.736834 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f62f8db-32f0-4801-a081-9b07dab6913b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:01.737115 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f62f8db-32f0-4801-a081-9b07dab6913b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:01.737396 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f62f8db-32f0-4801-a081-9b07dab6913b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:01.738552 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f62f8db-32f0-4801-a081-9b07dab6913b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:01.740010 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f62f8db-32f0-4801-a081-9b07dab6913b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:01.741601 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 43/82] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:44:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:01.742354 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f62f8db-32f0-4801-a081-9b07dab6913b None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:01.744076 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71af237-13c1-47a4-b045-1ac78419f6d9 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:01.777551 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778600c9-e2b0-4dc0-b029-39ecd56f1491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:01.778201 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778600c9-e2b0-4dc0-b029-39ecd56f1491 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:01.778488 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778600c9-e2b0-4dc0-b029-39ecd56f1491 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:01.778766 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778600c9-e2b0-4dc0-b029-39ecd56f1491 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:01.779602 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778600c9-e2b0-4dc0-b029-39ecd56f1491 None admin] RBAC: Adding query filter params (role.id=43106c664e1b40de984c0c62701149c8, scope.project.id=3663edc029e8431d904a228a9e5755a0, user.id=281b2f886b53468d808994f33b5f60ed) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:01.779938 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778600c9-e2b0-4dc0-b029-39ecd56f1491 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:01.781357 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778600c9-e2b0-4dc0-b029-39ecd56f1491 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:01.789667 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 40/83] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Jun 10 12:44:01 2022] GET /identity/v3/role_assignments?user.id=281b2f886b53468d808994f33b5f60ed&scope.project.id=3663edc029e8431d904a228a9e5755a0&role.id=43106c664e1b40de984c0c62701149c8 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:01.790452 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778600c9-e2b0-4dc0-b029-39ecd56f1491 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:01.850493 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f62f8db-32f0-4801-a081-9b07dab6913b None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:01.853182 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:01.890118 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff290a75-991c-4396-ab1a-f517ecc30eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:01.891226 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff290a75-991c-4396-ab1a-f517ecc30eaf None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:01.891518 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff290a75-991c-4396-ab1a-f517ecc30eaf None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:01.891802 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff290a75-991c-4396-ab1a-f517ecc30eaf None admin] PATH_INFO: `/v3/roles/member` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:01.905114 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff290a75-991c-4396-ab1a-f517ecc30eaf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:44:01.915481 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-ff290a75-991c-4396-ab1a-f517ecc30eaf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 10 12:44:01.915481 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:44:01.915481 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:44:01.915481 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:44:01.915481 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:44:01.915481 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:44:01.915481 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:44:01.915481 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:44:01.915481 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:44:01.915481 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:44:01.915481 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:44:01.915481 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:44:01.915481 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:44:01.915481 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:44:01.915481 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:44:01.915481 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:44:01.916707 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:44:01.916707 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:44:01.916707 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:44:01.916707 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:44:01.916707 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:44:01.916707 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:44:01.916707 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:44:01.916707 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:44:01.916707 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:44:01.916707 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:44:01.916707 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:44:01.916707 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:44:01.916707 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:44:01.916707 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:44:01.916707 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:44:01.916707 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:44:01.923704 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:44:01.923704 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:44:01.923704 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:44:01.923704 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:44:01.923704 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:44:01.923704 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:44:01.923704 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:44:01.923704 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:44:01.923704 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:44:01.923704 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:44:01.923704 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 10 12:44:01.923704 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:44:01.923704 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 44/84] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Jun 10 12:44:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:44:01.923704 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff290a75-991c-4396-ab1a-f517ecc30eaf None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:01.926864 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778600c9-e2b0-4dc0-b029-39ecd56f1491 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:01.944232 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:01.965039 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2611a9bb-d8c3-413e-b344-e86b30136f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:01.966404 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2611a9bb-d8c3-413e-b344-e86b30136f50 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:01.966404 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2611a9bb-d8c3-413e-b344-e86b30136f50 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:01.966650 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2611a9bb-d8c3-413e-b344-e86b30136f50 None admin] PATH_INFO: `/v3/users/281b2f886b53468d808994f33b5f60ed` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:01.967841 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2611a9bb-d8c3-413e-b344-e86b30136f50 None admin] RBAC: Authorizing `identity:get_user(user_id=281b2f886b53468d808994f33b5f60ed)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:01.969416 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2611a9bb-d8c3-413e-b344-e86b30136f50 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:01.971245 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 41/85] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Jun 10 12:44:01 2022] GET /identity/v3/users/281b2f886b53468d808994f33b5f60ed => generated 275 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:01.976270 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2611a9bb-d8c3-413e-b344-e86b30136f50 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:02.033394 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff290a75-991c-4396-ab1a-f517ecc30eaf None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:02.067657 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27652961-a26a-4e12-84b5-0a979a73679c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:02.068458 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27652961-a26a-4e12-84b5-0a979a73679c None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:02.068842 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27652961-a26a-4e12-84b5-0a979a73679c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:02.069220 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27652961-a26a-4e12-84b5-0a979a73679c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:02.083677 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2611a9bb-d8c3-413e-b344-e86b30136f50 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:02.103786 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:02.119998 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcad20af-fde0-4b08-a044-33674f64c213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:02.120739 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcad20af-fde0-4b08-a044-33674f64c213 None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:02.121023 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcad20af-fde0-4b08-a044-33674f64c213 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:02.121575 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcad20af-fde0-4b08-a044-33674f64c213 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:02.122013 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcad20af-fde0-4b08-a044-33674f64c213 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:02.123494 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcad20af-fde0-4b08-a044-33674f64c213 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:02.144215 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27652961-a26a-4e12-84b5-0a979a73679c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:02.145680 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27652961-a26a-4e12-84b5-0a979a73679c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:02.220016 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 42/86] 10.222.0.27 () {60 vars in 1132 bytes} [Fri Jun 10 12:44:01 2022] POST /identity/v3/users => generated 289 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 12:44:02.221295 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcad20af-fde0-4b08-a044-33674f64c213 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:02.258661 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 45/87] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Jun 10 12:44:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 10 12:44:02.259773 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27652961-a26a-4e12-84b5-0a979a73679c None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:02.280282 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:02.331331 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:02.348245 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcad20af-fde0-4b08-a044-33674f64c213 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:02.373526 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27652961-a26a-4e12-84b5-0a979a73679c None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:02.387109 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d018dbe6-c3a4-432e-abc3-b55f7232a43e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:02.387874 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d018dbe6-c3a4-432e-abc3-b55f7232a43e None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:02.388188 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d018dbe6-c3a4-432e-abc3-b55f7232a43e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:02.388560 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d018dbe6-c3a4-432e-abc3-b55f7232a43e None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:02.389322 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d018dbe6-c3a4-432e-abc3-b55f7232a43e None admin] RBAC: Adding query filter params (name=member) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:02.389636 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d018dbe6-c3a4-432e-abc3-b55f7232a43e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:02.391058 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d018dbe6-c3a4-432e-abc3-b55f7232a43e None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:02.401216 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 43/88] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Jun 10 12:44:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:02.402611 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d018dbe6-c3a4-432e-abc3-b55f7232a43e None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:02.412742 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec14729c-ff92-4079-af65-f260cab4fcd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:02.413413 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec14729c-ff92-4079-af65-f260cab4fcd4 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:02.413707 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec14729c-ff92-4079-af65-f260cab4fcd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:02.413992 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec14729c-ff92-4079-af65-f260cab4fcd4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:02.498538 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec14729c-ff92-4079-af65-f260cab4fcd4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:02.500457 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec14729c-ff92-4079-af65-f260cab4fcd4 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:02.531833 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d018dbe6-c3a4-432e-abc3-b55f7232a43e None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:02.533423 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:02.578977 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a7b9c4-6035-405f-ad2d-3a9975ab51bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:02.579855 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a7b9c4-6035-405f-ad2d-3a9975ab51bb None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:02.580214 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a7b9c4-6035-405f-ad2d-3a9975ab51bb None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:02.580499 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a7b9c4-6035-405f-ad2d-3a9975ab51bb None admin] PATH_INFO: `/v3/projects/3663edc029e8431d904a228a9e5755a0` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:02.581725 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a7b9c4-6035-405f-ad2d-3a9975ab51bb None admin] RBAC: Authorizing `identity:get_project(project_id=3663edc029e8431d904a228a9e5755a0)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:02.583199 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a7b9c4-6035-405f-ad2d-3a9975ab51bb None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:02.583560 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:02.585094 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 44/89] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Jun 10 12:44:02 2022] GET /identity/v3/projects/3663edc029e8431d904a228a9e5755a0 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:02.585755 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a7b9c4-6035-405f-ad2d-3a9975ab51bb None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:02.634638 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 46/90] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Jun 10 12:44:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 10 12:44:02.635261 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec14729c-ff92-4079-af65-f260cab4fcd4 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:02.733630 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a7b9c4-6035-405f-ad2d-3a9975ab51bb None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:02.753348 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec14729c-ff92-4079-af65-f260cab4fcd4 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:02.802600 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c559c97-d3b5-427f-91b2-963ee7b44fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:02.803491 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c559c97-d3b5-427f-91b2-963ee7b44fff None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:02.803776 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c559c97-d3b5-427f-91b2-963ee7b44fff None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:02.804077 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c559c97-d3b5-427f-91b2-963ee7b44fff None admin] PATH_INFO: `/v3/users/281b2f886b53468d808994f33b5f60ed` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:02.810464 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c559c97-d3b5-427f-91b2-963ee7b44fff None admin] RBAC: Authorizing `identity:get_user(user_id=281b2f886b53468d808994f33b5f60ed)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:02.815366 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c559c97-d3b5-427f-91b2-963ee7b44fff None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:02.816689 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9a1520-5418-453a-a7b3-62ca0094855b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:02.817351 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9a1520-5418-453a-a7b3-62ca0094855b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:02.817634 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9a1520-5418-453a-a7b3-62ca0094855b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:02.817911 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9a1520-5418-453a-a7b3-62ca0094855b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:02.819985 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 45/91] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Jun 10 12:44:02 2022] GET /identity/v3/users/281b2f886b53468d808994f33b5f60ed => generated 275 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:02.820867 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c559c97-d3b5-427f-91b2-963ee7b44fff None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:02.821469 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9a1520-5418-453a-a7b3-62ca0094855b None admin] RBAC: Adding query filter params (role.id=b43d5a8247e244869c2c70f986f42e8a, scope.project.id=3663edc029e8431d904a228a9e5755a0, user.id=85098428641c41389cc157b4c057067c) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:02.821793 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9a1520-5418-453a-a7b3-62ca0094855b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:02.823297 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9a1520-5418-453a-a7b3-62ca0094855b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:02.838832 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 47/92] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Jun 10 12:44:02 2022] GET /identity/v3/role_assignments?user.id=85098428641c41389cc157b4c057067c&scope.project.id=3663edc029e8431d904a228a9e5755a0&role.id=b43d5a8247e244869c2c70f986f42e8a => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:02.842221 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9a1520-5418-453a-a7b3-62ca0094855b None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:02.916367 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:02.929329 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:02.991307 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9a1520-5418-453a-a7b3-62ca0094855b None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:03.031112 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd9e773-2ab1-4dd8-a01a-dc43f8e1038e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:03.031781 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd9e773-2ab1-4dd8-a01a-dc43f8e1038e None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:03.032099 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd9e773-2ab1-4dd8-a01a-dc43f8e1038e None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:03.032385 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd9e773-2ab1-4dd8-a01a-dc43f8e1038e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:03.080397 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c559c97-d3b5-427f-91b2-963ee7b44fff None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:03.109138 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd9e773-2ab1-4dd8-a01a-dc43f8e1038e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:03.110639 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd9e773-2ab1-4dd8-a01a-dc43f8e1038e None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:03.146090 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf593bcf-60cd-46e5-a301-34ed03831829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:03.146981 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf593bcf-60cd-46e5-a301-34ed03831829 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:03.147266 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf593bcf-60cd-46e5-a301-34ed03831829 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:03.158939 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf593bcf-60cd-46e5-a301-34ed03831829 None admin] PATH_INFO: `/v3/projects/3663edc029e8431d904a228a9e5755a0` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:03.158939 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf593bcf-60cd-46e5-a301-34ed03831829 None admin] RBAC: Authorizing `identity:get_project(project_id=3663edc029e8431d904a228a9e5755a0)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:03.159235 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:03.159317 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf593bcf-60cd-46e5-a301-34ed03831829 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:03.159317 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 46/93] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Jun 10 12:44:02 2022] GET /identity/v3/projects/3663edc029e8431d904a228a9e5755a0 => generated 298 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:03.161369 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf593bcf-60cd-46e5-a301-34ed03831829 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:03.273431 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 48/94] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Jun 10 12:44:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 10 12:44:03.283031 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd9e773-2ab1-4dd8-a01a-dc43f8e1038e None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:03.321022 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf593bcf-60cd-46e5-a301-34ed03831829 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:03.329501 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:03.365000 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6787f4f3-c01f-48ca-af80-68c4da6cf6c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:03.365673 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6787f4f3-c01f-48ca-af80-68c4da6cf6c2 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:03.365964 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6787f4f3-c01f-48ca-af80-68c4da6cf6c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:03.366246 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6787f4f3-c01f-48ca-af80-68c4da6cf6c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:03.368042 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6787f4f3-c01f-48ca-af80-68c4da6cf6c2 None admin] RBAC: Adding query filter params (role.id=2d71ee7c2a544fe5ad77d92b24280efe, scope.project.id=3663edc029e8431d904a228a9e5755a0, user.id=281b2f886b53468d808994f33b5f60ed) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:03.368365 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6787f4f3-c01f-48ca-af80-68c4da6cf6c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:03.369800 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6787f4f3-c01f-48ca-af80-68c4da6cf6c2 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:03.382412 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 47/95] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Jun 10 12:44:03 2022] GET /identity/v3/role_assignments?user.id=281b2f886b53468d808994f33b5f60ed&scope.project.id=3663edc029e8431d904a228a9e5755a0&role.id=2d71ee7c2a544fe5ad77d92b24280efe => generated 259 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:03.477707 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd9e773-2ab1-4dd8-a01a-dc43f8e1038e None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:03.516555 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:03.532598 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42de57a2-c9c2-49d5-81aa-fb03ed39a26b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:03.532598 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42de57a2-c9c2-49d5-81aa-fb03ed39a26b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:03.532598 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42de57a2-c9c2-49d5-81aa-fb03ed39a26b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:03.532598 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42de57a2-c9c2-49d5-81aa-fb03ed39a26b None admin] PATH_INFO: `/v3/projects/dc9aa6385a6f44c6b86ffe510a6bc114` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:03.533435 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42de57a2-c9c2-49d5-81aa-fb03ed39a26b None admin] RBAC: Authorizing `identity:get_project(project_id=dc9aa6385a6f44c6b86ffe510a6bc114)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:03.534906 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42de57a2-c9c2-49d5-81aa-fb03ed39a26b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:03.537092 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 49/96] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Jun 10 12:44:03 2022] GET /identity/v3/projects/dc9aa6385a6f44c6b86ffe510a6bc114 => generated 312 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 12:44:03.545280 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6787f4f3-c01f-48ca-af80-68c4da6cf6c2 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:03.686089 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6787f4f3-c01f-48ca-af80-68c4da6cf6c2 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:03.730427 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cb74f9-fdfc-41d8-8266-75aab6283134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:03.731150 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cb74f9-fdfc-41d8-8266-75aab6283134 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:03.731440 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cb74f9-fdfc-41d8-8266-75aab6283134 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:03.731721 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cb74f9-fdfc-41d8-8266-75aab6283134 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:03.732589 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cb74f9-fdfc-41d8-8266-75aab6283134 None admin] RBAC: Adding query filter params (role.id=43106c664e1b40de984c0c62701149c8, scope.project.id=dc9aa6385a6f44c6b86ffe510a6bc114, user.id=281b2f886b53468d808994f33b5f60ed) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:03.732964 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cb74f9-fdfc-41d8-8266-75aab6283134 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:03.737000 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cb74f9-fdfc-41d8-8266-75aab6283134 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:03.743333 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 48/97] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Jun 10 12:44:03 2022] GET /identity/v3/role_assignments?user.id=281b2f886b53468d808994f33b5f60ed&scope.project.id=dc9aa6385a6f44c6b86ffe510a6bc114&role.id=43106c664e1b40de984c0c62701149c8 => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:04.695466 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9df33d-2c34-450b-b77a-b9f896486cb1 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:04.695989 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9df33d-2c34-450b-b77a-b9f896486cb1 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:04.696064 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9df33d-2c34-450b-b77a-b9f896486cb1 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:04.697414 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 50/98] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:44:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:44:04.702822 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebb986e-6196-4882-b977-7059beccc6cd None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:04.703227 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebb986e-6196-4882-b977-7059beccc6cd None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:04.703393 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebb986e-6196-4882-b977-7059beccc6cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:04.724269 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-7ebb986e-6196-4882-b977-7059beccc6cd None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:44:04.725447 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-7ebb986e-6196-4882-b977-7059beccc6cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:44:04.766322 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:04.822113 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 49/99] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:44:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 10 12:44:04.860417 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9df33d-2c34-450b-b77a-b9f896486cb1 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:04.994182 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9df33d-2c34-450b-b77a-b9f896486cb1 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:05.039096 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c59e8bd-bf47-4b3e-84b5-acfbb41febb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:05.039923 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c59e8bd-bf47-4b3e-84b5-acfbb41febb4 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:05.040210 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c59e8bd-bf47-4b3e-84b5-acfbb41febb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:05.040488 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c59e8bd-bf47-4b3e-84b5-acfbb41febb4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:05.055443 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c59e8bd-bf47-4b3e-84b5-acfbb41febb4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:44:05.063101 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-5c59e8bd-bf47-4b3e-84b5-acfbb41febb4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 10 12:44:05.063101 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:44:05.063101 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:44:05.063101 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:44:05.063101 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:44:05.063101 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:44:05.063101 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:44:05.063101 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:44:05.063101 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:44:05.063101 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:44:05.063101 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:44:05.063101 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:44:05.063101 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:44:05.063101 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:44:05.063101 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:44:05.063101 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:44:05.063996 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:44:05.063996 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:44:05.063996 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:44:05.063996 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:44:05.063996 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:44:05.063996 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:44:05.063996 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:44:05.063996 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:44:05.063996 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:44:05.063996 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:44:05.063996 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:44:05.063996 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:44:05.063996 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:44:05.063996 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:44:05.063996 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:44:05.063996 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:44:05.064765 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:44:05.064765 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:44:05.064765 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:44:05.064765 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:44:05.064765 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:44:05.064765 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:44:05.064765 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:44:05.064765 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:44:05.064765 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:44:05.064765 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:44:05.064765 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 10 12:44:05.064765 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:44:05.068458 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 51/100] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Jun 10 12:44:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:44:05.076949 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebb986e-6196-4882-b977-7059beccc6cd None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:05.189755 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42833ca4-1bf2-4c51-89b1-a4010eac39fe None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:05.190119 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42833ca4-1bf2-4c51-89b1-a4010eac39fe None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:05.190407 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42833ca4-1bf2-4c51-89b1-a4010eac39fe None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:05.191934 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 52/101] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:44:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 10 12:44:05.197756 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b70db-4895-431c-8590-c64d03eeb006 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:05.198052 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b70db-4895-431c-8590-c64d03eeb006 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:05.198368 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b70db-4895-431c-8590-c64d03eeb006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:05.219171 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-298b70db-4895-431c-8590-c64d03eeb006 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:44:05.220657 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-298b70db-4895-431c-8590-c64d03eeb006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:44:05.239462 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebb986e-6196-4882-b977-7059beccc6cd None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:05.304734 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:05.322231 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 53/102] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:44:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 10 12:44:05.324227 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979ce822-56b9-4b16-808e-f313924393cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:05.324356 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3143b720-bc72-4b37-bd66-55fd73612a08 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:05.324572 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3143b720-bc72-4b37-bd66-55fd73612a08 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:05.327893 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3143b720-bc72-4b37-bd66-55fd73612a08 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:05.327993 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979ce822-56b9-4b16-808e-f313924393cc None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:05.327993 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979ce822-56b9-4b16-808e-f313924393cc None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:05.327993 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979ce822-56b9-4b16-808e-f313924393cc None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:05.328214 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 54/103] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:44:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:44:05.328281 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979ce822-56b9-4b16-808e-f313924393cc None admin] RBAC: Adding query filter params (name=member) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:05.328281 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979ce822-56b9-4b16-808e-f313924393cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:05.329465 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979ce822-56b9-4b16-808e-f313924393cc None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:05.336243 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2948d7a-7f9d-4d5e-844c-73d2be411235 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:05.336534 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2948d7a-7f9d-4d5e-844c-73d2be411235 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:05.336819 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2948d7a-7f9d-4d5e-844c-73d2be411235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:05.347425 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 50/104] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Jun 10 12:44:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:05.352999 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979ce822-56b9-4b16-808e-f313924393cc None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:05.355039 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-d2948d7a-7f9d-4d5e-844c-73d2be411235 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:44:05.356274 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d2948d7a-7f9d-4d5e-844c-73d2be411235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:44:05.415321 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:05.429835 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 55/105] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:44:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 10 12:44:05.430913 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2948d7a-7f9d-4d5e-844c-73d2be411235 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:05.541924 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2948d7a-7f9d-4d5e-844c-73d2be411235 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:05.559664 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:05.579462 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232e1958-6b4b-4afc-8df5-5c7a70c22e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:05.580744 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232e1958-6b4b-4afc-8df5-5c7a70c22e30 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:05.581246 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232e1958-6b4b-4afc-8df5-5c7a70c22e30 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:05.581732 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232e1958-6b4b-4afc-8df5-5c7a70c22e30 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:05.594345 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232e1958-6b4b-4afc-8df5-5c7a70c22e30 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:44:05.599623 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979ce822-56b9-4b16-808e-f313924393cc None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:05.602859 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-232e1958-6b4b-4afc-8df5-5c7a70c22e30 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 10 12:44:05.602859 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:44:05.602859 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:44:05.602859 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:44:05.602859 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:44:05.602859 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:44:05.602859 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:44:05.602859 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:44:05.602859 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:44:05.602859 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:44:05.602859 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:44:05.602859 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:44:05.602859 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:44:05.602859 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:44:05.602859 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:44:05.602859 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:44:05.603790 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:44:05.603790 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:44:05.603790 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:44:05.603790 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:44:05.603790 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:44:05.603790 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:44:05.603790 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:44:05.603790 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:44:05.603790 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:44:05.603790 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:44:05.603790 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:44:05.603790 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:44:05.603790 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:44:05.603790 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:44:05.603790 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:44:05.603790 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:44:05.604636 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:44:05.604636 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:44:05.604636 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:44:05.604636 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:44:05.604636 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:44:05.604636 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:44:05.604636 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:44:05.604636 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:44:05.604636 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:44:05.604636 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:44:05.604636 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 10 12:44:05.604636 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:44:05.607094 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 56/106] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Jun 10 12:44:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:44:05.608238 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232e1958-6b4b-4afc-8df5-5c7a70c22e30 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:05.689444 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ab8b76-8e5f-4708-aff1-2a9f610018f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:05.690341 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ab8b76-8e5f-4708-aff1-2a9f610018f8 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:05.690632 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ab8b76-8e5f-4708-aff1-2a9f610018f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:05.690916 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ab8b76-8e5f-4708-aff1-2a9f610018f8 None admin] PATH_INFO: `/v3/users/281b2f886b53468d808994f33b5f60ed` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:05.696639 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ab8b76-8e5f-4708-aff1-2a9f610018f8 None admin] RBAC: Authorizing `identity:get_user(user_id=281b2f886b53468d808994f33b5f60ed)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:05.698943 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ab8b76-8e5f-4708-aff1-2a9f610018f8 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:05.700357 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 51/107] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Jun 10 12:44:05 2022] GET /identity/v3/users/281b2f886b53468d808994f33b5f60ed => generated 275 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 12:44:05.701128 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ab8b76-8e5f-4708-aff1-2a9f610018f8 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:05.722406 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232e1958-6b4b-4afc-8df5-5c7a70c22e30 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:05.759991 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6274e558-9bd0-4eec-8758-6e54f6fbf51f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:05.761758 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6274e558-9bd0-4eec-8758-6e54f6fbf51f None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:05.762286 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6274e558-9bd0-4eec-8758-6e54f6fbf51f None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:05.762794 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6274e558-9bd0-4eec-8758-6e54f6fbf51f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:05.764245 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6274e558-9bd0-4eec-8758-6e54f6fbf51f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:05.765904 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6274e558-9bd0-4eec-8758-6e54f6fbf51f None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:05.767910 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 57/108] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:44:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:05.769039 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6274e558-9bd0-4eec-8758-6e54f6fbf51f None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:05.818205 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:05.884605 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6274e558-9bd0-4eec-8758-6e54f6fbf51f None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:05.951872 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:05.955463 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a495d75-7729-4408-8745-5fa777d5c169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:05.956373 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a495d75-7729-4408-8745-5fa777d5c169 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:05.956663 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a495d75-7729-4408-8745-5fa777d5c169 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:05.956949 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a495d75-7729-4408-8745-5fa777d5c169 None admin] PATH_INFO: `/v3/projects/3663edc029e8431d904a228a9e5755a0` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:05.959245 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a495d75-7729-4408-8745-5fa777d5c169 None admin] RBAC: Authorizing `identity:get_project(project_id=3663edc029e8431d904a228a9e5755a0)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:05.961498 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a495d75-7729-4408-8745-5fa777d5c169 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:05.963225 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 58/109] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Jun 10 12:44:05 2022] GET /identity/v3/projects/3663edc029e8431d904a228a9e5755a0 => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:05.970883 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ab8b76-8e5f-4708-aff1-2a9f610018f8 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:05.972422 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a495d75-7729-4408-8745-5fa777d5c169 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:06.041069 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2f5eea-e762-482f-bee7-1280e8143773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:06.042656 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2f5eea-e762-482f-bee7-1280e8143773 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:06.043165 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2f5eea-e762-482f-bee7-1280e8143773 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:06.043632 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2f5eea-e762-482f-bee7-1280e8143773 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:06.044594 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2f5eea-e762-482f-bee7-1280e8143773 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:06.050374 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2f5eea-e762-482f-bee7-1280e8143773 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:06.052061 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2f5eea-e762-482f-bee7-1280e8143773 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:06.071294 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 52/110] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Jun 10 12:44:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:06.072507 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2f5eea-e762-482f-bee7-1280e8143773 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:06.099269 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a495d75-7729-4408-8745-5fa777d5c169 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:06.120681 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:06.147177 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db55c13-3527-4f4b-8a80-16ea62ed5968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:06.148281 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db55c13-3527-4f4b-8a80-16ea62ed5968 None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:06.148814 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db55c13-3527-4f4b-8a80-16ea62ed5968 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:06.149294 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db55c13-3527-4f4b-8a80-16ea62ed5968 None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:06.150337 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db55c13-3527-4f4b-8a80-16ea62ed5968 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:06.152817 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db55c13-3527-4f4b-8a80-16ea62ed5968 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:06.198948 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2f5eea-e762-482f-bee7-1280e8143773 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:06.233625 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc056e0-0b9c-401a-87a8-34537cd98129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:06.234522 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc056e0-0b9c-401a-87a8-34537cd98129 None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:06.234808 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc056e0-0b9c-401a-87a8-34537cd98129 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:06.235089 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc056e0-0b9c-401a-87a8-34537cd98129 None admin] PATH_INFO: `/v3/projects/3663edc029e8431d904a228a9e5755a0/users/281b2f886b53468d808994f33b5f60ed/roles/43106c664e1b40de984c0c62701149c8` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:06.237133 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc056e0-0b9c-401a-87a8-34537cd98129 None admin] RBAC: Authorizing `identity:create_grant(project_id=3663edc029e8431d904a228a9e5755a0, user_id=281b2f886b53468d808994f33b5f60ed, role_id=43106c664e1b40de984c0c62701149c8)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:06.238577 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc056e0-0b9c-401a-87a8-34537cd98129 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:06.247494 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 59/111] 10.222.0.27 () {60 vars in 1141 bytes} [Fri Jun 10 12:44:05 2022] POST /identity/v3/projects => generated 302 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 12:44:06.252082 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d5681a-6c79-44b5-97b1-1ca8e3aaf084 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:06.252082 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d5681a-6c79-44b5-97b1-1ca8e3aaf084 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:06.252082 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d5681a-6c79-44b5-97b1-1ca8e3aaf084 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:06.252082 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 60/112] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:44:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:44:06.252082 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d5681a-6c79-44b5-97b1-1ca8e3aaf084 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:06.330246 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 53/113] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Jun 10 12:44:06 2022] PUT /identity/v3/projects/3663edc029e8431d904a228a9e5755a0/users/281b2f886b53468d808994f33b5f60ed/roles/43106c664e1b40de984c0c62701149c8 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 12:44:06.331555 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d13f30-0fc3-48fd-9bad-b7b04a5e5abf None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:06.331853 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d13f30-0fc3-48fd-9bad-b7b04a5e5abf None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:06.332161 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d13f30-0fc3-48fd-9bad-b7b04a5e5abf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:06.370663 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-c4d13f30-0fc3-48fd-9bad-b7b04a5e5abf None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:44:06.372114 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c4d13f30-0fc3-48fd-9bad-b7b04a5e5abf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:44:06.393964 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:06.435551 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d5681a-6c79-44b5-97b1-1ca8e3aaf084 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:06.492563 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9f0a6a-a08e-48df-b206-c5e32908882d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:06.494040 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9f0a6a-a08e-48df-b206-c5e32908882d None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:06.494249 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9f0a6a-a08e-48df-b206-c5e32908882d None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:06.494519 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9f0a6a-a08e-48df-b206-c5e32908882d None admin] PATH_INFO: `/v3/users/b1832bf1dd584457ace688490fd59d10` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:06.520336 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9f0a6a-a08e-48df-b206-c5e32908882d None admin] RBAC: Authorizing `identity:get_user(user_id=b1832bf1dd584457ace688490fd59d10)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:06.521846 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9f0a6a-a08e-48df-b206-c5e32908882d None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:06.523716 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 61/114] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Jun 10 12:44:06 2022] GET /identity/v3/users/b1832bf1dd584457ace688490fd59d10 => generated 289 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:06.530355 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9f0a6a-a08e-48df-b206-c5e32908882d None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:06.538793 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 54/115] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:44:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 10 12:44:06.572279 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d13f30-0fc3-48fd-9bad-b7b04a5e5abf None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:06.578795 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:06.664486 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9f0a6a-a08e-48df-b206-c5e32908882d None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:06.701628 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60612ed1-2eb7-476f-976b-37286eaea56b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:06.702621 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60612ed1-2eb7-476f-976b-37286eaea56b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:06.703119 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60612ed1-2eb7-476f-976b-37286eaea56b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:06.703614 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60612ed1-2eb7-476f-976b-37286eaea56b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:06.720809 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d13f30-0fc3-48fd-9bad-b7b04a5e5abf None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:06.764531 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fc033b-a256-477c-9f75-d8b7f2b0dc5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:06.765340 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fc033b-a256-477c-9f75-d8b7f2b0dc5b None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:06.765624 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fc033b-a256-477c-9f75-d8b7f2b0dc5b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:06.765951 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fc033b-a256-477c-9f75-d8b7f2b0dc5b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:06.777999 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fc033b-a256-477c-9f75-d8b7f2b0dc5b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:44:06.785904 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-76fc033b-a256-477c-9f75-d8b7f2b0dc5b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 10 12:44:06.785904 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:44:06.785904 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:44:06.785904 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:44:06.785904 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:44:06.785904 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:44:06.785904 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:44:06.785904 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:44:06.785904 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:44:06.785904 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:44:06.785904 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:44:06.785904 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:44:06.785904 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:44:06.785904 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:44:06.785904 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:44:06.785904 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:44:06.786801 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:44:06.786801 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:44:06.786801 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:44:06.786801 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:44:06.786801 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:44:06.786801 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:44:06.786801 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:44:06.786801 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:44:06.786801 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:44:06.786801 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:44:06.786801 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:44:06.786801 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:44:06.786801 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:44:06.786801 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:44:06.786801 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:44:06.786801 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:44:06.787853 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:44:06.787853 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:44:06.787853 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:44:06.787853 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:44:06.787853 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:44:06.787853 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:44:06.787853 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:44:06.787853 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:44:06.787853 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:44:06.787853 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:44:06.787853 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 10 12:44:06.787853 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:44:06.787853 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 55/116] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Jun 10 12:44:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:44:06.789006 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0207a6da-b2fb-457f-b8f6-426ad671ee28 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:06.789006 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0207a6da-b2fb-457f-b8f6-426ad671ee28 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:06.789255 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0207a6da-b2fb-457f-b8f6-426ad671ee28 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:06.790571 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 56/117] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:44:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:44:06.795420 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0207a6da-b2fb-457f-b8f6-426ad671ee28 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:06.796267 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60612ed1-2eb7-476f-976b-37286eaea56b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:06.798677 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60612ed1-2eb7-476f-976b-37286eaea56b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:06.807029 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:06.834305 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:06.916734 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0207a6da-b2fb-457f-b8f6-426ad671ee28 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:06.940042 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 62/118] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Jun 10 12:44:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 10 12:44:06.940042 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed1577d-4e1e-4234-abc1-c1f35133b39a None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:06.940042 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed1577d-4e1e-4234-abc1-c1f35133b39a None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:06.940042 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed1577d-4e1e-4234-abc1-c1f35133b39a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:06.972102 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-3ed1577d-4e1e-4234-abc1-c1f35133b39a None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:44:06.974070 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ae72a5-6b51-4728-ab46-afd2f606d1b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:06.974747 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ae72a5-6b51-4728-ab46-afd2f606d1b2 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:06.975069 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ae72a5-6b51-4728-ab46-afd2f606d1b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:06.975350 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ae72a5-6b51-4728-ab46-afd2f606d1b2 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:06.976176 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ae72a5-6b51-4728-ab46-afd2f606d1b2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:06.976491 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ae72a5-6b51-4728-ab46-afd2f606d1b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:06.977978 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-3ed1577d-4e1e-4234-abc1-c1f35133b39a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:44:06.979759 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ae72a5-6b51-4728-ab46-afd2f606d1b2 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:07.003170 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 57/119] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Jun 10 12:44:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:07.005179 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ae72a5-6b51-4728-ab46-afd2f606d1b2 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:07.091417 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:07.110077 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 63/120] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:44:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 10 12:44:07.111292 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed1577d-4e1e-4234-abc1-c1f35133b39a None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:07.132042 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ae72a5-6b51-4728-ab46-afd2f606d1b2 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:07.175797 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2abc8b-84ca-4582-907c-d53634f8cade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:07.176805 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2abc8b-84ca-4582-907c-d53634f8cade None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:07.177145 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2abc8b-84ca-4582-907c-d53634f8cade None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:07.177669 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2abc8b-84ca-4582-907c-d53634f8cade None admin] PATH_INFO: `/v3/projects/3663edc029e8431d904a228a9e5755a0` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:07.179091 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2abc8b-84ca-4582-907c-d53634f8cade None admin] RBAC: Authorizing `identity:get_project(project_id=3663edc029e8431d904a228a9e5755a0)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:07.180758 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2abc8b-84ca-4582-907c-d53634f8cade None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:07.183486 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 58/121] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Jun 10 12:44:07 2022] GET /identity/v3/projects/3663edc029e8431d904a228a9e5755a0 => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:07.184786 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c66b1b-ab1a-48da-b8ac-e014e3ac47ae None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:07.185105 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c66b1b-ab1a-48da-b8ac-e014e3ac47ae None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:07.185396 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c66b1b-ab1a-48da-b8ac-e014e3ac47ae None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:07.186921 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:07.188020 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 59/122] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:44:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:44:07.193256 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c66b1b-ab1a-48da-b8ac-e014e3ac47ae None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:07.238666 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed1577d-4e1e-4234-abc1-c1f35133b39a None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:07.274613 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d31c33-f895-4d15-a907-2ded721bd816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:07.275506 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d31c33-f895-4d15-a907-2ded721bd816 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:07.275796 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d31c33-f895-4d15-a907-2ded721bd816 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:07.276116 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d31c33-f895-4d15-a907-2ded721bd816 None admin] PATH_INFO: `/v3/users/85098428641c41389cc157b4c057067c` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:07.277367 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d31c33-f895-4d15-a907-2ded721bd816 None admin] RBAC: Authorizing `identity:get_user(user_id=85098428641c41389cc157b4c057067c)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:07.278850 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d31c33-f895-4d15-a907-2ded721bd816 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:07.280850 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 64/123] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Jun 10 12:44:07 2022] GET /identity/v3/users/85098428641c41389cc157b4c057067c => generated 247 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:07.281700 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d31c33-f895-4d15-a907-2ded721bd816 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:07.314279 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c66b1b-ab1a-48da-b8ac-e014e3ac47ae None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:07.352492 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895de1c1-9eed-492b-9b8a-18a49c84b8d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:07.353601 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895de1c1-9eed-492b-9b8a-18a49c84b8d7 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:07.354061 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895de1c1-9eed-492b-9b8a-18a49c84b8d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:07.354623 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895de1c1-9eed-492b-9b8a-18a49c84b8d7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:07.357843 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:07.368397 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895de1c1-9eed-492b-9b8a-18a49c84b8d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:44:07.378470 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-895de1c1-9eed-492b-9b8a-18a49c84b8d7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 10 12:44:07.378470 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:44:07.378470 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:44:07.378470 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:44:07.378470 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:44:07.378470 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:44:07.378470 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:44:07.378470 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:44:07.378470 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:44:07.378470 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:44:07.378470 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:44:07.378470 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:44:07.378470 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:44:07.378470 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:44:07.378470 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:44:07.378470 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:44:07.379331 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:44:07.379331 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:44:07.379331 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:44:07.379331 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:44:07.379331 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:44:07.379331 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:44:07.379331 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:44:07.379331 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:44:07.379331 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:44:07.379331 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:44:07.379331 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:44:07.379331 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:44:07.379331 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:44:07.379331 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:44:07.379331 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:44:07.379331 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:44:07.380145 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:44:07.380145 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:44:07.380145 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:44:07.380145 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:44:07.380145 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:44:07.380145 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:44:07.380145 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:44:07.380145 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:44:07.380145 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:44:07.380145 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:44:07.380145 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 10 12:44:07.380145 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:44:07.382743 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 60/124] 10.222.0.27 () {56 vars in 1114 bytes} [Fri Jun 10 12:44:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:44:07.384241 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35658bd-40a3-4589-854c-26f81d323c1f None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:07.384720 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35658bd-40a3-4589-854c-26f81d323c1f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:07.386664 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35658bd-40a3-4589-854c-26f81d323c1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:07.386960 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d31c33-f895-4d15-a907-2ded721bd816 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:07.404755 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-f35658bd-40a3-4589-854c-26f81d323c1f None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:44:07.406292 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f35658bd-40a3-4589-854c-26f81d323c1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:44:07.407806 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:07.422340 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bffcf6-decd-4baf-883f-890a89931400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:07.423020 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bffcf6-decd-4baf-883f-890a89931400 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:07.423289 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bffcf6-decd-4baf-883f-890a89931400 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:07.423574 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bffcf6-decd-4baf-883f-890a89931400 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:07.424678 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bffcf6-decd-4baf-883f-890a89931400 None admin] RBAC: Adding query filter params (role.id=7d6c4b2adb9d43bca1a6573e8f7a9381, scope.project.id=3663edc029e8431d904a228a9e5755a0, user.id=b1832bf1dd584457ace688490fd59d10) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:07.424783 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bffcf6-decd-4baf-883f-890a89931400 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:07.426348 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bffcf6-decd-4baf-883f-890a89931400 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:07.431945 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 65/125] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Jun 10 12:44:07 2022] GET /identity/v3/role_assignments?user.id=b1832bf1dd584457ace688490fd59d10&scope.project.id=3663edc029e8431d904a228a9e5755a0&role.id=7d6c4b2adb9d43bca1a6573e8f7a9381 => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:07.433284 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bffcf6-decd-4baf-883f-890a89931400 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:07.508538 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 61/126] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:44:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 10 12:44:07.509707 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35658bd-40a3-4589-854c-26f81d323c1f None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:07.539616 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bffcf6-decd-4baf-883f-890a89931400 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:07.607302 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e54d3c4-ca98-452b-a711-f37f6defc27f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:07.608855 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e54d3c4-ca98-452b-a711-f37f6defc27f None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:07.609146 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e54d3c4-ca98-452b-a711-f37f6defc27f None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:07.609431 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e54d3c4-ca98-452b-a711-f37f6defc27f None admin] PATH_INFO: `/v3/projects/3663edc029e8431d904a228a9e5755a0` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:07.611474 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e54d3c4-ca98-452b-a711-f37f6defc27f None admin] RBAC: Authorizing `identity:get_project(project_id=3663edc029e8431d904a228a9e5755a0)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:07.613835 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e54d3c4-ca98-452b-a711-f37f6defc27f None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:07.615599 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 66/127] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Jun 10 12:44:07 2022] GET /identity/v3/projects/3663edc029e8431d904a228a9e5755a0 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:07.618065 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e54d3c4-ca98-452b-a711-f37f6defc27f None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:07.708829 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35658bd-40a3-4589-854c-26f81d323c1f None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:07.719550 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:07.739836 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:07.749960 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8439dde-bb6f-4f76-a3fd-dfec748ba8af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:07.750886 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8439dde-bb6f-4f76-a3fd-dfec748ba8af None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:07.751372 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8439dde-bb6f-4f76-a3fd-dfec748ba8af None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:07.751821 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8439dde-bb6f-4f76-a3fd-dfec748ba8af None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:07.752768 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8439dde-bb6f-4f76-a3fd-dfec748ba8af None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:07.753245 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8439dde-bb6f-4f76-a3fd-dfec748ba8af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:07.754803 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8439dde-bb6f-4f76-a3fd-dfec748ba8af None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:07.763690 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 62/128] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Jun 10 12:44:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:07.764852 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8439dde-bb6f-4f76-a3fd-dfec748ba8af None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:07.773754 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e54d3c4-ca98-452b-a711-f37f6defc27f None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:07.827151 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604241bc-6585-4df3-ad8f-e87ec3359300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:07.827989 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604241bc-6585-4df3-ad8f-e87ec3359300 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:07.828281 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604241bc-6585-4df3-ad8f-e87ec3359300 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:07.828571 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604241bc-6585-4df3-ad8f-e87ec3359300 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:07.848430 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604241bc-6585-4df3-ad8f-e87ec3359300 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:44:07.862555 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-604241bc-6585-4df3-ad8f-e87ec3359300 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 10 12:44:07.862555 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:44:07.862555 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:44:07.862555 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:44:07.862555 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:44:07.862555 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:44:07.862555 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:44:07.862555 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:44:07.862555 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:44:07.862555 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:44:07.862555 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:44:07.862555 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:44:07.862555 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:44:07.862555 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:44:07.862555 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:44:07.862555 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:44:07.863458 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:44:07.863458 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:44:07.863458 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:44:07.863458 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:44:07.863458 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:44:07.863458 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:44:07.863458 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:44:07.863458 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:44:07.863458 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:44:07.863458 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:44:07.863458 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:44:07.863458 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:44:07.863458 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:44:07.863458 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:44:07.863458 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:44:07.863458 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:44:07.864291 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:44:07.864291 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:44:07.864291 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:44:07.864291 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:44:07.864291 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:44:07.864291 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:44:07.864291 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:44:07.864291 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:44:07.864291 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:44:07.864291 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:44:07.864291 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 10 12:44:07.864291 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:44:07.864291 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 67/129] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Jun 10 12:44:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:44:07.864291 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604241bc-6585-4df3-ad8f-e87ec3359300 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:07.959315 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8439dde-bb6f-4f76-a3fd-dfec748ba8af None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:08.033449 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c90cbf-1a84-4d5e-acd8-6c5b2b47ccd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:08.034344 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c90cbf-1a84-4d5e-acd8-6c5b2b47ccd1 None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:08.034630 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c90cbf-1a84-4d5e-acd8-6c5b2b47ccd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:08.034913 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c90cbf-1a84-4d5e-acd8-6c5b2b47ccd1 None admin] PATH_INFO: `/v3/projects/3663edc029e8431d904a228a9e5755a0/users/85098428641c41389cc157b4c057067c/roles/b43d5a8247e244869c2c70f986f42e8a` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:08.038701 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604241bc-6585-4df3-ad8f-e87ec3359300 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:08.044679 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c90cbf-1a84-4d5e-acd8-6c5b2b47ccd1 None admin] RBAC: Authorizing `identity:create_grant(project_id=3663edc029e8431d904a228a9e5755a0, user_id=85098428641c41389cc157b4c057067c, role_id=b43d5a8247e244869c2c70f986f42e8a)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:08.046149 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c90cbf-1a84-4d5e-acd8-6c5b2b47ccd1 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:08.052557 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:08.074049 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd229268-aae2-428a-aea2-510eb0bb3ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:08.074936 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd229268-aae2-428a-aea2-510eb0bb3ec8 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:08.075230 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd229268-aae2-428a-aea2-510eb0bb3ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:08.075513 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd229268-aae2-428a-aea2-510eb0bb3ec8 None admin] PATH_INFO: `/v3/users/281b2f886b53468d808994f33b5f60ed` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:08.076846 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd229268-aae2-428a-aea2-510eb0bb3ec8 None admin] RBAC: Authorizing `identity:get_user(user_id=281b2f886b53468d808994f33b5f60ed)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:08.078334 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd229268-aae2-428a-aea2-510eb0bb3ec8 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:08.081310 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 68/130] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Jun 10 12:44:07 2022] GET /identity/v3/users/281b2f886b53468d808994f33b5f60ed => generated 275 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:08.082606 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd229268-aae2-428a-aea2-510eb0bb3ec8 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:08.169868 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 63/131] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Jun 10 12:44:07 2022] PUT /identity/v3/projects/3663edc029e8431d904a228a9e5755a0/users/85098428641c41389cc157b4c057067c/roles/b43d5a8247e244869c2c70f986f42e8a => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 12:44:08.171181 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c90cbf-1a84-4d5e-acd8-6c5b2b47ccd1 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:08.195290 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd229268-aae2-428a-aea2-510eb0bb3ec8 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:08.238311 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1264f4-64da-4ee9-b40c-ae2d927c154e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:08.238920 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1264f4-64da-4ee9-b40c-ae2d927c154e None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:08.239208 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1264f4-64da-4ee9-b40c-ae2d927c154e None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:08.239482 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1264f4-64da-4ee9-b40c-ae2d927c154e None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:08.240296 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1264f4-64da-4ee9-b40c-ae2d927c154e None admin] RBAC: Adding query filter params (name=member) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:08.240608 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1264f4-64da-4ee9-b40c-ae2d927c154e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:08.243639 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1264f4-64da-4ee9-b40c-ae2d927c154e None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:08.259920 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 69/132] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Jun 10 12:44:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:08.266344 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1264f4-64da-4ee9-b40c-ae2d927c154e None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:08.286622 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:08.353131 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:08.412108 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c90cbf-1a84-4d5e-acd8-6c5b2b47ccd1 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:08.469543 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1264f4-64da-4ee9-b40c-ae2d927c154e None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:08.500776 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d91054-a563-412a-891b-1ad8fc873455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:08.501581 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d91054-a563-412a-891b-1ad8fc873455 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:08.501875 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d91054-a563-412a-891b-1ad8fc873455 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:08.502157 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d91054-a563-412a-891b-1ad8fc873455 None admin] PATH_INFO: `/v3/projects/3663edc029e8431d904a228a9e5755a0` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:08.504009 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d91054-a563-412a-891b-1ad8fc873455 None admin] RBAC: Authorizing `identity:get_project(project_id=3663edc029e8431d904a228a9e5755a0)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:08.505582 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d91054-a563-412a-891b-1ad8fc873455 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:08.507406 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 64/133] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Jun 10 12:44:08 2022] GET /identity/v3/projects/3663edc029e8431d904a228a9e5755a0 => generated 298 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:08.508344 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6a90cd-a8e2-4625-9e1a-80885121feea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:08.509304 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6a90cd-a8e2-4625-9e1a-80885121feea None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:08.509596 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6a90cd-a8e2-4625-9e1a-80885121feea None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:08.509917 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6a90cd-a8e2-4625-9e1a-80885121feea None admin] PATH_INFO: `/v3/users/281b2f886b53468d808994f33b5f60ed` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:08.511690 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6a90cd-a8e2-4625-9e1a-80885121feea None admin] RBAC: Authorizing `identity:get_user(user_id=281b2f886b53468d808994f33b5f60ed)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:08.513177 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6a90cd-a8e2-4625-9e1a-80885121feea None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:08.513682 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d91054-a563-412a-891b-1ad8fc873455 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:08.514801 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 70/134] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Jun 10 12:44:08 2022] GET /identity/v3/users/281b2f886b53468d808994f33b5f60ed => generated 275 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 12:44:08.520201 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6a90cd-a8e2-4625-9e1a-80885121feea None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:08.627656 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6a90cd-a8e2-4625-9e1a-80885121feea None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:08.639751 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:08.668564 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f143681c-6e83-4796-9e9a-1a10c58834a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:08.669400 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f143681c-6e83-4796-9e9a-1a10c58834a7 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:08.669695 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f143681c-6e83-4796-9e9a-1a10c58834a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:08.669987 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f143681c-6e83-4796-9e9a-1a10c58834a7 None admin] PATH_INFO: `/v3/projects/dc9aa6385a6f44c6b86ffe510a6bc114` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:08.673422 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f143681c-6e83-4796-9e9a-1a10c58834a7 None admin] RBAC: Authorizing `identity:get_project(project_id=dc9aa6385a6f44c6b86ffe510a6bc114)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:08.674885 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f143681c-6e83-4796-9e9a-1a10c58834a7 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:08.677474 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 71/135] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Jun 10 12:44:08 2022] GET /identity/v3/projects/dc9aa6385a6f44c6b86ffe510a6bc114 => generated 312 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:08.691451 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f143681c-6e83-4796-9e9a-1a10c58834a7 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:08.741856 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d91054-a563-412a-891b-1ad8fc873455 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:08.769333 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:08.801382 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca175b84-0101-4162-aad9-1be645bc4711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:08.802536 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca175b84-0101-4162-aad9-1be645bc4711 None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:08.802914 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca175b84-0101-4162-aad9-1be645bc4711 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:08.803314 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca175b84-0101-4162-aad9-1be645bc4711 None admin] PATH_INFO: `/v3/projects/3663edc029e8431d904a228a9e5755a0/users/281b2f886b53468d808994f33b5f60ed/roles/2d71ee7c2a544fe5ad77d92b24280efe` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:08.806004 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca175b84-0101-4162-aad9-1be645bc4711 None admin] RBAC: Authorizing `identity:create_grant(project_id=3663edc029e8431d904a228a9e5755a0, user_id=281b2f886b53468d808994f33b5f60ed, role_id=2d71ee7c2a544fe5ad77d92b24280efe)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:08.808119 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca175b84-0101-4162-aad9-1be645bc4711 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:08.812066 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f143681c-6e83-4796-9e9a-1a10c58834a7 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:08.848952 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415f179c-4543-476b-9aca-821c55895670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:08.850557 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415f179c-4543-476b-9aca-821c55895670 None admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:08.851010 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415f179c-4543-476b-9aca-821c55895670 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:08.851478 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415f179c-4543-476b-9aca-821c55895670 None admin] PATH_INFO: `/v3/projects/dc9aa6385a6f44c6b86ffe510a6bc114/users/281b2f886b53468d808994f33b5f60ed/roles/43106c664e1b40de984c0c62701149c8` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:08.853721 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415f179c-4543-476b-9aca-821c55895670 None admin] RBAC: Authorizing `identity:create_grant(project_id=dc9aa6385a6f44c6b86ffe510a6bc114, user_id=281b2f886b53468d808994f33b5f60ed, role_id=43106c664e1b40de984c0c62701149c8)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:08.855519 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415f179c-4543-476b-9aca-821c55895670 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:08.896601 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 65/136] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Jun 10 12:44:08 2022] PUT /identity/v3/projects/3663edc029e8431d904a228a9e5755a0/users/281b2f886b53468d808994f33b5f60ed/roles/2d71ee7c2a544fe5ad77d92b24280efe => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 10 12:44:08.926771 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 72/137] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Jun 10 12:44:08 2022] PUT /identity/v3/projects/dc9aa6385a6f44c6b86ffe510a6bc114/users/281b2f886b53468d808994f33b5f60ed/roles/43106c664e1b40de984c0c62701149c8 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 10 12:44:09.046673 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24e1797-1f16-4397-9a45-b06759feff6d None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:09.047173 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24e1797-1f16-4397-9a45-b06759feff6d None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:09.047173 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24e1797-1f16-4397-9a45-b06759feff6d None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:09.048954 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 66/138] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:44:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:44:09.055255 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1598eb52-bb06-41da-ba0c-3a9e58245524 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:09.055971 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1598eb52-bb06-41da-ba0c-3a9e58245524 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:09.056455 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1598eb52-bb06-41da-ba0c-3a9e58245524 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:09.083120 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-1598eb52-bb06-41da-ba0c-3a9e58245524 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:44:09.084926 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-1598eb52-bb06-41da-ba0c-3a9e58245524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:44:09.126401 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:09.171018 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 73/139] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:44:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 10 12:44:09.199488 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24e1797-1f16-4397-9a45-b06759feff6d None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:09.216902 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01c0a2d-d4f3-48da-8364-af928e82ae7a None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:09.217859 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01c0a2d-d4f3-48da-8364-af928e82ae7a None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:09.218533 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01c0a2d-d4f3-48da-8364-af928e82ae7a None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:09.220296 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 74/140] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:44:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:44:09.226801 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06de702a-0f1e-41e8-be43-5c67c0314e1e None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:09.227545 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06de702a-0f1e-41e8-be43-5c67c0314e1e None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:09.228054 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06de702a-0f1e-41e8-be43-5c67c0314e1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:09.249507 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-06de702a-0f1e-41e8-be43-5c67c0314e1e None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:44:09.251673 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-06de702a-0f1e-41e8-be43-5c67c0314e1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:44:09.328787 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 75/141] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:44:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jun 10 12:44:09.358760 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06de702a-0f1e-41e8-be43-5c67c0314e1e None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:09.464445 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:09.468392 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06de702a-0f1e-41e8-be43-5c67c0314e1e None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:09.496535 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24e1797-1f16-4397-9a45-b06759feff6d None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:09.505060 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bde866-710a-41b4-b3b4-9094f7462360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:09.506137 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bde866-710a-41b4-b3b4-9094f7462360 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:09.506690 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bde866-710a-41b4-b3b4-9094f7462360 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:09.507198 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bde866-710a-41b4-b3b4-9094f7462360 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:09.508635 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bde866-710a-41b4-b3b4-9094f7462360 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:09.510289 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bde866-710a-41b4-b3b4-9094f7462360 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:09.512194 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 76/142] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:44:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 12:44:09.517698 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bde866-710a-41b4-b3b4-9094f7462360 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:09.553903 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dbca10-cca9-4985-a6a2-fc05ad0892b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:09.554683 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dbca10-cca9-4985-a6a2-fc05ad0892b5 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:09.554976 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dbca10-cca9-4985-a6a2-fc05ad0892b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:09.555244 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dbca10-cca9-4985-a6a2-fc05ad0892b5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:09.580026 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dbca10-cca9-4985-a6a2-fc05ad0892b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:44:09.587711 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-f0dbca10-cca9-4985-a6a2-fc05ad0892b5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 10 12:44:09.587711 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:44:09.587711 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:44:09.587711 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:44:09.587711 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:44:09.587711 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:44:09.587711 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:44:09.587711 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:44:09.587711 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:44:09.587711 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:44:09.587711 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:44:09.587711 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:44:09.587711 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:44:09.587711 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:44:09.587711 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:44:09.587711 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:44:09.589172 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:44:09.589172 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:44:09.589172 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:44:09.589172 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:44:09.589172 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:44:09.589172 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:44:09.589172 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:44:09.589172 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:44:09.589172 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:44:09.589172 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:44:09.589172 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:44:09.589172 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:44:09.589172 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:44:09.589172 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:44:09.589172 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:44:09.589172 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:44:09.589960 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:44:09.589960 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:44:09.589960 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:44:09.589960 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:44:09.589960 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:44:09.589960 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:44:09.589960 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:44:09.589960 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:44:09.589960 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:44:09.589960 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:44:09.589960 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 10 12:44:09.589960 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:44:09.589960 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 67/143] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Jun 10 12:44:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 391 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:44:09.595243 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dbca10-cca9-4985-a6a2-fc05ad0892b5 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:09.660242 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:09.679125 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bde866-710a-41b4-b3b4-9094f7462360 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:09.737597 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9192b0-eb99-4204-94d1-bd2ab52174df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:09.738598 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9192b0-eb99-4204-94d1-bd2ab52174df None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:09.739044 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9192b0-eb99-4204-94d1-bd2ab52174df None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:09.739480 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9192b0-eb99-4204-94d1-bd2ab52174df None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:09.740412 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9192b0-eb99-4204-94d1-bd2ab52174df None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:09.742119 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9192b0-eb99-4204-94d1-bd2ab52174df None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:09.747258 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dbca10-cca9-4985-a6a2-fc05ad0892b5 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:09.783559 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2d8348-a604-419e-86a4-51f2839863ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:09.784318 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2d8348-a604-419e-86a4-51f2839863ea None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:09.784635 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2d8348-a604-419e-86a4-51f2839863ea None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:09.784899 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2d8348-a604-419e-86a4-51f2839863ea None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:09.785665 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2d8348-a604-419e-86a4-51f2839863ea None admin] RBAC: Adding query filter params (name=member) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:09.785979 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2d8348-a604-419e-86a4-51f2839863ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:09.787618 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2d8348-a604-419e-86a4-51f2839863ea None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:09.797937 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 68/144] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Jun 10 12:44:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:09.803251 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2d8348-a604-419e-86a4-51f2839863ea None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:09.834422 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 77/145] 10.222.0.27 () {60 vars in 1132 bytes} [Fri Jun 10 12:44:09 2022] POST /identity/v3/users => generated 283 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 10 12:44:09.907458 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:09.910454 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2d8348-a604-419e-86a4-51f2839863ea None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:09.946092 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15304dc5-e817-4b20-b974-3358bd559450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:09.946978 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15304dc5-e817-4b20-b974-3358bd559450 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:09.947256 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15304dc5-e817-4b20-b974-3358bd559450 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:09.947541 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15304dc5-e817-4b20-b974-3358bd559450 None admin] PATH_INFO: `/v3/users/281b2f886b53468d808994f33b5f60ed` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:09.949302 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15304dc5-e817-4b20-b974-3358bd559450 None admin] RBAC: Authorizing `identity:get_user(user_id=281b2f886b53468d808994f33b5f60ed)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:09.950740 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15304dc5-e817-4b20-b974-3358bd559450 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:09.952528 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 69/146] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Jun 10 12:44:09 2022] GET /identity/v3/users/281b2f886b53468d808994f33b5f60ed => generated 275 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 12:44:09.958092 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9192b0-eb99-4204-94d1-bd2ab52174df None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:10.006173 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:10.106227 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9192b0-eb99-4204-94d1-bd2ab52174df None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:10.147418 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874c057d-c6b8-4465-8145-c71dc3cbd243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:10.148144 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874c057d-c6b8-4465-8145-c71dc3cbd243 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:10.148431 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874c057d-c6b8-4465-8145-c71dc3cbd243 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:10.148711 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874c057d-c6b8-4465-8145-c71dc3cbd243 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:10.222320 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874c057d-c6b8-4465-8145-c71dc3cbd243 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:10.223807 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874c057d-c6b8-4465-8145-c71dc3cbd243 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:10.248218 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:10.328524 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 78/147] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Jun 10 12:44:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 10 12:44:10.334371 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15304dc5-e817-4b20-b974-3358bd559450 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:10.439549 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15304dc5-e817-4b20-b974-3358bd559450 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:10.474995 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86231402-0f6c-458b-b438-15fbe3814032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:10.475922 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86231402-0f6c-458b-b438-15fbe3814032 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:10.476188 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86231402-0f6c-458b-b438-15fbe3814032 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:10.476469 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86231402-0f6c-458b-b438-15fbe3814032 None admin] PATH_INFO: `/v3/projects/3663edc029e8431d904a228a9e5755a0` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:10.477671 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86231402-0f6c-458b-b438-15fbe3814032 None admin] RBAC: Authorizing `identity:get_project(project_id=3663edc029e8431d904a228a9e5755a0)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:10.479124 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86231402-0f6c-458b-b438-15fbe3814032 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:10.481408 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 70/148] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Jun 10 12:44:10 2022] GET /identity/v3/projects/3663edc029e8431d904a228a9e5755a0 => generated 298 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 12:44:10.487207 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874c057d-c6b8-4465-8145-c71dc3cbd243 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:10.593615 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874c057d-c6b8-4465-8145-c71dc3cbd243 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:10.618406 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:10.630585 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590a82a6-990b-40fe-ac0d-b7137bab2e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:10.633925 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590a82a6-990b-40fe-ac0d-b7137bab2e3c None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:10.633925 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590a82a6-990b-40fe-ac0d-b7137bab2e3c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:10.633925 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590a82a6-990b-40fe-ac0d-b7137bab2e3c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:10.633925 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590a82a6-990b-40fe-ac0d-b7137bab2e3c None admin] RBAC: Adding query filter params (role.id=43106c664e1b40de984c0c62701149c8, scope.project.id=3663edc029e8431d904a228a9e5755a0, user.id=281b2f886b53468d808994f33b5f60ed) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:10.633925 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590a82a6-990b-40fe-ac0d-b7137bab2e3c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:10.634648 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590a82a6-990b-40fe-ac0d-b7137bab2e3c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:10.642413 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 79/149] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Jun 10 12:44:10 2022] GET /identity/v3/role_assignments?user.id=281b2f886b53468d808994f33b5f60ed&scope.project.id=3663edc029e8431d904a228a9e5755a0&role.id=43106c664e1b40de984c0c62701149c8 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:10.703040 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00f5322-0760-4356-909f-2c07fa196401 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:10.703311 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00f5322-0760-4356-909f-2c07fa196401 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:10.703531 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00f5322-0760-4356-909f-2c07fa196401 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:10.704936 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 71/150] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:44:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:44:10.711003 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07619733-fd11-423c-ab3e-387a4cd4674f None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:10.711319 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07619733-fd11-423c-ab3e-387a4cd4674f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:10.711561 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07619733-fd11-423c-ab3e-387a4cd4674f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:10.732999 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-07619733-fd11-423c-ab3e-387a4cd4674f None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:44:10.734241 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-07619733-fd11-423c-ab3e-387a4cd4674f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:44:10.821443 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 80/151] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:44:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 10 12:44:10.864377 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00f5322-0760-4356-909f-2c07fa196401 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:10.907050 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:10.983072 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00f5322-0760-4356-909f-2c07fa196401 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:11.024388 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313a967d-6e22-459a-a13b-85bf31a54d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:11.025212 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313a967d-6e22-459a-a13b-85bf31a54d87 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:11.025499 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313a967d-6e22-459a-a13b-85bf31a54d87 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:11.025786 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313a967d-6e22-459a-a13b-85bf31a54d87 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:11.038822 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313a967d-6e22-459a-a13b-85bf31a54d87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:44:11.046792 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-313a967d-6e22-459a-a13b-85bf31a54d87 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 10 12:44:11.046792 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:44:11.046792 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:44:11.046792 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:44:11.046792 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:44:11.046792 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:44:11.046792 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:44:11.046792 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:44:11.046792 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:44:11.046792 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:44:11.046792 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:44:11.046792 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:44:11.046792 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:44:11.046792 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:44:11.046792 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:44:11.046792 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:44:11.047669 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:44:11.047669 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:44:11.047669 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:44:11.047669 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:44:11.047669 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:44:11.047669 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:44:11.047669 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:44:11.047669 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:44:11.047669 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:44:11.047669 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:44:11.047669 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:44:11.047669 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:44:11.047669 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:44:11.047669 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:44:11.047669 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:44:11.047669 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:44:11.048478 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:44:11.048478 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:44:11.048478 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:44:11.048478 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:44:11.048478 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:44:11.048478 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:44:11.048478 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:44:11.048478 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:44:11.048478 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:44:11.048478 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:44:11.048478 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 10 12:44:11.048478 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:44:11.048478 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 72/152] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Jun 10 12:44:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 10 12:44:11.054037 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07619733-fd11-423c-ab3e-387a4cd4674f None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:11.160689 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07619733-fd11-423c-ab3e-387a4cd4674f None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:11.178987 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:11.197534 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552ff3be-736d-4364-a2bf-423eeafe8946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:11.198797 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552ff3be-736d-4364-a2bf-423eeafe8946 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:11.199080 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552ff3be-736d-4364-a2bf-423eeafe8946 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:11.199410 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552ff3be-736d-4364-a2bf-423eeafe8946 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:11.200267 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552ff3be-736d-4364-a2bf-423eeafe8946 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:11.200626 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552ff3be-736d-4364-a2bf-423eeafe8946 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:11.202033 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552ff3be-736d-4364-a2bf-423eeafe8946 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:11.210827 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 81/153] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Jun 10 12:44:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:11.216437 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313a967d-6e22-459a-a13b-85bf31a54d87 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:11.337738 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:11.344438 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313a967d-6e22-459a-a13b-85bf31a54d87 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:11.383451 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0223fb58-2d9d-4f04-bc83-269aa3cf43e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:11.384405 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0223fb58-2d9d-4f04-bc83-269aa3cf43e3 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:11.384695 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0223fb58-2d9d-4f04-bc83-269aa3cf43e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:11.384958 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0223fb58-2d9d-4f04-bc83-269aa3cf43e3 None admin] PATH_INFO: `/v3/users/b1832bf1dd584457ace688490fd59d10` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:11.386384 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0223fb58-2d9d-4f04-bc83-269aa3cf43e3 None admin] RBAC: Authorizing `identity:get_user(user_id=b1832bf1dd584457ace688490fd59d10)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:11.388040 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0223fb58-2d9d-4f04-bc83-269aa3cf43e3 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:11.389962 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 73/154] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Jun 10 12:44:11 2022] GET /identity/v3/users/b1832bf1dd584457ace688490fd59d10 => generated 289 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:11.395635 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552ff3be-736d-4364-a2bf-423eeafe8946 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:11.445568 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abf0683-5cec-4740-85c4-4a213442a64b None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:11.445882 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abf0683-5cec-4740-85c4-4a213442a64b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:11.446124 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abf0683-5cec-4740-85c4-4a213442a64b None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:11.447759 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 74/155] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:44:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:44:11.457151 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ded0f9-98ee-47e2-9be6-5b0a54649dfc None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:11.457437 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ded0f9-98ee-47e2-9be6-5b0a54649dfc None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:11.457717 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ded0f9-98ee-47e2-9be6-5b0a54649dfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:11.486217 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-69ded0f9-98ee-47e2-9be6-5b0a54649dfc None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:44:11.487420 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-69ded0f9-98ee-47e2-9be6-5b0a54649dfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:44:11.552379 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552ff3be-736d-4364-a2bf-423eeafe8946 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:11.600746 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 75/156] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:44:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 10 12:44:11.607025 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d70554-dc75-4955-86c7-043322b270ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:11.608166 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d70554-dc75-4955-86c7-043322b270ed None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:11.608642 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d70554-dc75-4955-86c7-043322b270ed None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:11.609107 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d70554-dc75-4955-86c7-043322b270ed None admin] PATH_INFO: `/v3/projects/3663edc029e8431d904a228a9e5755a0` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:11.616780 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d70554-dc75-4955-86c7-043322b270ed None admin] RBAC: Authorizing `identity:get_project(project_id=3663edc029e8431d904a228a9e5755a0)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:11.618484 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d70554-dc75-4955-86c7-043322b270ed None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:11.620637 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 82/157] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Jun 10 12:44:11 2022] GET /identity/v3/projects/3663edc029e8431d904a228a9e5755a0 => generated 298 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:11.626328 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ded0f9-98ee-47e2-9be6-5b0a54649dfc None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:11.642336 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d70554-dc75-4955-86c7-043322b270ed None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:11.741712 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:11.760561 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ded0f9-98ee-47e2-9be6-5b0a54649dfc None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:11.772255 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:11.800392 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0990def8-cbee-4195-b4b6-d516651c24f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:11.801579 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0990def8-cbee-4195-b4b6-d516651c24f1 None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:11.801870 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0990def8-cbee-4195-b4b6-d516651c24f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:11.802149 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0990def8-cbee-4195-b4b6-d516651c24f1 None admin] PATH_INFO: `/v3/projects/3663edc029e8431d904a228a9e5755a0/users/b1832bf1dd584457ace688490fd59d10/roles/7d6c4b2adb9d43bca1a6573e8f7a9381` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:11.804443 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0990def8-cbee-4195-b4b6-d516651c24f1 None admin] RBAC: Authorizing `identity:create_grant(project_id=3663edc029e8431d904a228a9e5755a0, user_id=b1832bf1dd584457ace688490fd59d10, role_id=7d6c4b2adb9d43bca1a6573e8f7a9381)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:11.805895 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0990def8-cbee-4195-b4b6-d516651c24f1 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:11.864029 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d70554-dc75-4955-86c7-043322b270ed None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:11.881070 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 76/158] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Jun 10 12:44:11 2022] PUT /identity/v3/projects/3663edc029e8431d904a228a9e5755a0/users/b1832bf1dd584457ace688490fd59d10/roles/7d6c4b2adb9d43bca1a6573e8f7a9381 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 12:44:11.926648 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605860fa-9f85-421f-a087-cb6fc987e137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:11.927531 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605860fa-9f85-421f-a087-cb6fc987e137 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:11.927817 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605860fa-9f85-421f-a087-cb6fc987e137 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:11.928131 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605860fa-9f85-421f-a087-cb6fc987e137 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:11.943604 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605860fa-9f85-421f-a087-cb6fc987e137 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:44:11.951570 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-605860fa-9f85-421f-a087-cb6fc987e137 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 10 12:44:11.951570 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:44:11.951570 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:44:11.951570 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:44:11.951570 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:44:11.951570 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:44:11.951570 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:44:11.951570 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:44:11.951570 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:44:11.951570 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:44:11.951570 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:44:11.951570 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:44:11.951570 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:44:11.951570 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:44:11.951570 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:44:11.951570 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:44:11.952527 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:44:11.952527 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:44:11.952527 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:44:11.952527 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:44:11.952527 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:44:11.952527 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:44:11.952527 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:44:11.952527 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:44:11.952527 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:44:11.952527 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:44:11.952527 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:44:11.952527 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:44:11.952527 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:44:11.952527 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:44:11.952527 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:44:11.952527 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:44:11.953295 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:44:11.953295 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:44:11.953295 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:44:11.953295 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:44:11.953295 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:44:11.953295 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:44:11.953295 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:44:11.953295 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:44:11.953295 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:44:11.953295 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:44:11.953295 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 10 12:44:11.953295 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:44:11.953295 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 83/159] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Jun 10 12:44:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 10 12:44:11.959024 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0990def8-cbee-4195-b4b6-d516651c24f1 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:12.115148 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0990def8-cbee-4195-b4b6-d516651c24f1 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:12.162989 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd5a4e9-2649-4d18-94ba-c50fadfee91e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:12.164068 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd5a4e9-2649-4d18-94ba-c50fadfee91e None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:12.164545 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd5a4e9-2649-4d18-94ba-c50fadfee91e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:12.165009 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd5a4e9-2649-4d18-94ba-c50fadfee91e None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:12.165983 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd5a4e9-2649-4d18-94ba-c50fadfee91e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:12.166501 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd5a4e9-2649-4d18-94ba-c50fadfee91e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:12.168161 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd5a4e9-2649-4d18-94ba-c50fadfee91e None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:12.184117 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 77/160] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Jun 10 12:44:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:12.189599 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605860fa-9f85-421f-a087-cb6fc987e137 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:12.304151 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bb452a-b72a-4c24-ac5c-fde03a8a7b02 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:12.304840 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bb452a-b72a-4c24-ac5c-fde03a8a7b02 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:12.305319 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bb452a-b72a-4c24-ac5c-fde03a8a7b02 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:12.306882 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 78/161] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:44:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 10 12:44:12.312885 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae7e5cf-4e37-4dbe-af5c-c04a5a6043f2 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:12.313460 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae7e5cf-4e37-4dbe-af5c-c04a5a6043f2 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:12.313954 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae7e5cf-4e37-4dbe-af5c-c04a5a6043f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:12.336503 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605860fa-9f85-421f-a087-cb6fc987e137 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:12.346475 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-4ae7e5cf-4e37-4dbe-af5c-c04a5a6043f2 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:44:12.348355 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:12.348490 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-4ae7e5cf-4e37-4dbe-af5c-c04a5a6043f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:44:12.372563 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:12.386449 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bb4370-cb26-442f-81c2-d937c48fd3b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:12.387341 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bb4370-cb26-442f-81c2-d937c48fd3b9 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:12.387622 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bb4370-cb26-442f-81c2-d937c48fd3b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:12.387916 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bb4370-cb26-442f-81c2-d937c48fd3b9 None admin] PATH_INFO: `/v3/users/85098428641c41389cc157b4c057067c` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:12.389262 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bb4370-cb26-442f-81c2-d937c48fd3b9 None admin] RBAC: Authorizing `identity:get_user(user_id=85098428641c41389cc157b4c057067c)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:12.395726 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bb4370-cb26-442f-81c2-d937c48fd3b9 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:12.397507 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 84/162] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Jun 10 12:44:12 2022] GET /identity/v3/users/85098428641c41389cc157b4c057067c => generated 247 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:12.405023 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bb4370-cb26-442f-81c2-d937c48fd3b9 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:12.522231 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 79/163] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:44:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 10 12:44:12.572044 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57526183-8e64-4992-a0bc-7d07e92fc5f5 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:12.572705 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57526183-8e64-4992-a0bc-7d07e92fc5f5 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:12.573869 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57526183-8e64-4992-a0bc-7d07e92fc5f5 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:12.575461 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 80/164] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:44:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 10 12:44:12.587105 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6b0513-8195-4219-9eea-836154aaf5a9 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:12.587682 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6b0513-8195-4219-9eea-836154aaf5a9 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:12.588203 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6b0513-8195-4219-9eea-836154aaf5a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:12.600706 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bb4370-cb26-442f-81c2-d937c48fd3b9 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:12.632976 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-0b6b0513-8195-4219-9eea-836154aaf5a9 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:44:12.634098 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0b6b0513-8195-4219-9eea-836154aaf5a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:44:12.637329 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5ebdea-a751-4f4d-bcda-c848a539bbf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:12.638238 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5ebdea-a751-4f4d-bcda-c848a539bbf3 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:12.638731 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5ebdea-a751-4f4d-bcda-c848a539bbf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:12.639203 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5ebdea-a751-4f4d-bcda-c848a539bbf3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:12.716923 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:12.744171 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5ebdea-a751-4f4d-bcda-c848a539bbf3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:12.746287 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5ebdea-a751-4f4d-bcda-c848a539bbf3 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:12.834490 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 81/165] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:44:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 10 12:44:12.835425 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6b0513-8195-4219-9eea-836154aaf5a9 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:12.850263 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 85/166] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Jun 10 12:44:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 10 12:44:12.857201 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5ebdea-a751-4f4d-bcda-c848a539bbf3 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:12.989148 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:13.005230 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:13.045971 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6b0513-8195-4219-9eea-836154aaf5a9 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:13.064622 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5ebdea-a751-4f4d-bcda-c848a539bbf3 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:13.130123 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6227d5d8-4311-414e-84b3-991de9c15a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:13.131195 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6227d5d8-4311-414e-84b3-991de9c15a18 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:13.131475 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6227d5d8-4311-414e-84b3-991de9c15a18 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:13.131747 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6227d5d8-4311-414e-84b3-991de9c15a18 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:13.137939 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d9c72a-c79b-4e4a-b161-8d5fb784e74b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:13.138808 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d9c72a-c79b-4e4a-b161-8d5fb784e74b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:13.139094 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d9c72a-c79b-4e4a-b161-8d5fb784e74b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:13.139419 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d9c72a-c79b-4e4a-b161-8d5fb784e74b None admin] PATH_INFO: `/v3/projects/3663edc029e8431d904a228a9e5755a0` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:13.144619 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d9c72a-c79b-4e4a-b161-8d5fb784e74b None admin] RBAC: Authorizing `identity:get_project(project_id=3663edc029e8431d904a228a9e5755a0)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:13.146083 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d9c72a-c79b-4e4a-b161-8d5fb784e74b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:13.147818 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 86/167] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Jun 10 12:44:12 2022] GET /identity/v3/projects/3663edc029e8431d904a228a9e5755a0 => generated 298 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:13.148602 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d9c72a-c79b-4e4a-b161-8d5fb784e74b None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:13.161065 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6227d5d8-4311-414e-84b3-991de9c15a18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:44:13.175814 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-6227d5d8-4311-414e-84b3-991de9c15a18 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 10 12:44:13.175814 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:44:13.175814 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:44:13.175814 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:44:13.175814 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:44:13.175814 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:44:13.175814 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:44:13.175814 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:44:13.175814 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:44:13.175814 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:44:13.175814 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:44:13.175814 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:44:13.175814 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:44:13.175814 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:44:13.175814 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:44:13.175814 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:44:13.176883 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:44:13.176883 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:44:13.176883 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:44:13.176883 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:44:13.176883 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:44:13.176883 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:44:13.176883 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:44:13.176883 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:44:13.176883 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:44:13.176883 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:44:13.176883 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:44:13.176883 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:44:13.176883 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:44:13.176883 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:44:13.176883 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:44:13.176883 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:44:13.177733 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:44:13.177733 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:44:13.177733 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:44:13.177733 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:44:13.177733 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:44:13.177733 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:44:13.177733 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:44:13.177733 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:44:13.177733 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:44:13.177733 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:44:13.177733 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 10 12:44:13.177733 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:44:13.177733 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 82/168] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Jun 10 12:44:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:44:13.178395 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6227d5d8-4311-414e-84b3-991de9c15a18 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:13.365215 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6227d5d8-4311-414e-84b3-991de9c15a18 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:13.399926 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d9c72a-c79b-4e4a-b161-8d5fb784e74b None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:13.436573 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc5196a-d12a-4697-bd70-a653418c48e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:13.437248 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc5196a-d12a-4697-bd70-a653418c48e1 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:13.437533 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc5196a-d12a-4697-bd70-a653418c48e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:13.437813 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc5196a-d12a-4697-bd70-a653418c48e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:13.438659 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc5196a-d12a-4697-bd70-a653418c48e1 None admin] RBAC: Adding query filter params (role.id=b43d5a8247e244869c2c70f986f42e8a, scope.project.id=3663edc029e8431d904a228a9e5755a0, user.id=85098428641c41389cc157b4c057067c) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:13.438971 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc5196a-d12a-4697-bd70-a653418c48e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:13.440457 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc5196a-d12a-4697-bd70-a653418c48e1 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:13.443679 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:13.451068 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 83/169] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Jun 10 12:44:13 2022] GET /identity/v3/role_assignments?user.id=85098428641c41389cc157b4c057067c&scope.project.id=3663edc029e8431d904a228a9e5755a0&role.id=b43d5a8247e244869c2c70f986f42e8a => generated 608 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:13.451772 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc5196a-d12a-4697-bd70-a653418c48e1 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:13.468744 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cdb9b3-4ae8-432e-a3b5-7714356a7a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:13.469583 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cdb9b3-4ae8-432e-a3b5-7714356a7a1f None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:13.469868 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cdb9b3-4ae8-432e-a3b5-7714356a7a1f None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:13.470144 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cdb9b3-4ae8-432e-a3b5-7714356a7a1f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:13.492554 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cdb9b3-4ae8-432e-a3b5-7714356a7a1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:44:13.500740 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-29cdb9b3-4ae8-432e-a3b5-7714356a7a1f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 10 12:44:13.500740 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:44:13.500740 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:44:13.500740 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:44:13.500740 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:44:13.500740 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:44:13.500740 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:44:13.500740 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:44:13.500740 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:44:13.500740 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:44:13.500740 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:44:13.500740 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:44:13.500740 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:44:13.500740 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:44:13.500740 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:44:13.500740 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:44:13.501720 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:44:13.501720 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:44:13.501720 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:44:13.501720 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:44:13.501720 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:44:13.501720 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:44:13.501720 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:44:13.501720 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:44:13.501720 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:44:13.501720 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:44:13.501720 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:44:13.501720 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:44:13.501720 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:44:13.501720 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:44:13.501720 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:44:13.501720 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:44:13.502573 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:44:13.502573 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:44:13.502573 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:44:13.502573 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:44:13.502573 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:44:13.502573 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:44:13.502573 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:44:13.502573 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:44:13.502573 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:44:13.502573 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:44:13.502573 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 10 12:44:13.502573 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:44:13.508001 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 87/170] 10.222.0.27 () {56 vars in 1114 bytes} [Fri Jun 10 12:44:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:44:13.513279 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cdb9b3-4ae8-432e-a3b5-7714356a7a1f None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:13.523906 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:13.613219 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc5196a-d12a-4697-bd70-a653418c48e1 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:13.679545 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db61407-9f16-4644-849f-a705df5a67be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:13.679545 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db61407-9f16-4644-849f-a705df5a67be None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:13.679545 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db61407-9f16-4644-849f-a705df5a67be None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:13.679545 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db61407-9f16-4644-849f-a705df5a67be None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:13.679959 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db61407-9f16-4644-849f-a705df5a67be None admin] RBAC: Adding query filter params (name=member) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:13.680036 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cdb9b3-4ae8-432e-a3b5-7714356a7a1f None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:13.680134 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db61407-9f16-4644-849f-a705df5a67be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:13.681458 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db61407-9f16-4644-849f-a705df5a67be None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:13.699371 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 84/171] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Jun 10 12:44:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:13.705094 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db61407-9f16-4644-849f-a705df5a67be None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:13.744610 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27234720-ce00-434c-85fe-de69b6968f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:13.745335 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27234720-ce00-434c-85fe-de69b6968f96 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:13.745622 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27234720-ce00-434c-85fe-de69b6968f96 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:13.745900 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27234720-ce00-434c-85fe-de69b6968f96 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:13.746663 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27234720-ce00-434c-85fe-de69b6968f96 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:13.746968 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27234720-ce00-434c-85fe-de69b6968f96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:13.748407 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27234720-ce00-434c-85fe-de69b6968f96 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:13.762261 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 88/172] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Jun 10 12:44:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 12:44:13.768115 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27234720-ce00-434c-85fe-de69b6968f96 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:13.800964 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:13.864370 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db61407-9f16-4644-849f-a705df5a67be None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:13.876206 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:13.928637 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28151d78-31d4-4e83-b16c-38929d1a0a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:13.929852 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28151d78-31d4-4e83-b16c-38929d1a0a5f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:13.930337 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28151d78-31d4-4e83-b16c-38929d1a0a5f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:13.930787 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28151d78-31d4-4e83-b16c-38929d1a0a5f None admin] PATH_INFO: `/v3/users/281b2f886b53468d808994f33b5f60ed` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:13.932821 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28151d78-31d4-4e83-b16c-38929d1a0a5f None admin] RBAC: Authorizing `identity:get_user(user_id=281b2f886b53468d808994f33b5f60ed)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:13.934465 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28151d78-31d4-4e83-b16c-38929d1a0a5f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:13.939212 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27234720-ce00-434c-85fe-de69b6968f96 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:13.940258 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 85/173] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Jun 10 12:44:13 2022] GET /identity/v3/users/281b2f886b53468d808994f33b5f60ed => generated 275 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:13.945857 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28151d78-31d4-4e83-b16c-38929d1a0a5f None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:14.002602 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074481cb-0131-43bc-befc-7780176c54b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:14.003724 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074481cb-0131-43bc-befc-7780176c54b6 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:14.004092 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074481cb-0131-43bc-befc-7780176c54b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:14.004435 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074481cb-0131-43bc-befc-7780176c54b6 None admin] PATH_INFO: `/v3/users/281b2f886b53468d808994f33b5f60ed` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:14.005803 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074481cb-0131-43bc-befc-7780176c54b6 None admin] RBAC: Authorizing `identity:get_user(user_id=281b2f886b53468d808994f33b5f60ed)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:14.007368 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074481cb-0131-43bc-befc-7780176c54b6 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:14.009352 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 89/174] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Jun 10 12:44:13 2022] GET /identity/v3/users/281b2f886b53468d808994f33b5f60ed => generated 275 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:14.017614 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074481cb-0131-43bc-befc-7780176c54b6 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:14.066501 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28151d78-31d4-4e83-b16c-38929d1a0a5f None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:14.103889 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:14.117530 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf847ba-6994-4c6d-8422-51619e23e2bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:14.118536 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf847ba-6994-4c6d-8422-51619e23e2bd None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:14.119036 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf847ba-6994-4c6d-8422-51619e23e2bd None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:14.119520 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf847ba-6994-4c6d-8422-51619e23e2bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:14.203049 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074481cb-0131-43bc-befc-7780176c54b6 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:14.223585 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf847ba-6994-4c6d-8422-51619e23e2bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:14.225118 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf847ba-6994-4c6d-8422-51619e23e2bd None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:14.265736 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:14.274439 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f1f76f-6db4-4067-b140-5de3f11054f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:14.285532 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f1f76f-6db4-4067-b140-5de3f11054f1 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:14.285532 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f1f76f-6db4-4067-b140-5de3f11054f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:14.285532 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f1f76f-6db4-4067-b140-5de3f11054f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:14.399892 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f1f76f-6db4-4067-b140-5de3f11054f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:14.401947 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f1f76f-6db4-4067-b140-5de3f11054f1 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:14.409794 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 86/175] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Jun 10 12:44:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 10 12:44:14.411647 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ef4c15-bcb9-464a-8c65-a32632c61e91 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:14.412801 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ef4c15-bcb9-464a-8c65-a32632c61e91 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:14.413238 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ef4c15-bcb9-464a-8c65-a32632c61e91 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:14.416319 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 87/176] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:44:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:44:14.421095 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ef4c15-bcb9-464a-8c65-a32632c61e91 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:14.484475 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:14.592597 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 90/177] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Jun 10 12:44:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 575 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 10 12:44:14.594331 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298f4f9a-6241-42a3-b705-aaf07c48c4fb None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:14.594765 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298f4f9a-6241-42a3-b705-aaf07c48c4fb None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:14.595187 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298f4f9a-6241-42a3-b705-aaf07c48c4fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:14.614058 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ef4c15-bcb9-464a-8c65-a32632c61e91 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:14.615752 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-298f4f9a-6241-42a3-b705-aaf07c48c4fb None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:44:14.617950 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-298f4f9a-6241-42a3-b705-aaf07c48c4fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:44:14.652459 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:14.653692 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aebf66f-67de-4861-a034-7b4fad1c3a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:14.655249 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aebf66f-67de-4861-a034-7b4fad1c3a0b None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:14.655541 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aebf66f-67de-4861-a034-7b4fad1c3a0b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:14.655823 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aebf66f-67de-4861-a034-7b4fad1c3a0b None admin] PATH_INFO: `/v3/projects/dc9aa6385a6f44c6b86ffe510a6bc114` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:14.657233 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aebf66f-67de-4861-a034-7b4fad1c3a0b None admin] RBAC: Authorizing `identity:get_project(project_id=dc9aa6385a6f44c6b86ffe510a6bc114)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:14.659319 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aebf66f-67de-4861-a034-7b4fad1c3a0b None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:14.661724 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 88/178] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Jun 10 12:44:14 2022] GET /identity/v3/projects/dc9aa6385a6f44c6b86ffe510a6bc114 => generated 312 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:14.663793 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bb08c0-482d-42f4-bd64-51cf96385c07 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:14.663793 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bb08c0-482d-42f4-bd64-51cf96385c07 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:14.664109 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bb08c0-482d-42f4-bd64-51cf96385c07 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:14.666194 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 89/179] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:44:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:44:14.671583 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bb08c0-482d-42f4-bd64-51cf96385c07 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:14.702783 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 91/180] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:44:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 10 12:44:14.704065 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298f4f9a-6241-42a3-b705-aaf07c48c4fb None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:14.793440 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:14.802304 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bb08c0-482d-42f4-bd64-51cf96385c07 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:14.824975 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298f4f9a-6241-42a3-b705-aaf07c48c4fb None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:14.845841 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047cc7b5-49b3-4404-95ad-76539e16e405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:14.846866 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047cc7b5-49b3-4404-95ad-76539e16e405 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:14.847441 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047cc7b5-49b3-4404-95ad-76539e16e405 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:14.847725 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047cc7b5-49b3-4404-95ad-76539e16e405 None admin] PATH_INFO: `/v3/projects/3663edc029e8431d904a228a9e5755a0` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:14.849000 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047cc7b5-49b3-4404-95ad-76539e16e405 None admin] RBAC: Authorizing `identity:get_project(project_id=3663edc029e8431d904a228a9e5755a0)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:14.850962 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047cc7b5-49b3-4404-95ad-76539e16e405 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:14.853436 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 90/181] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Jun 10 12:44:14 2022] GET /identity/v3/projects/3663edc029e8431d904a228a9e5755a0 => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:14.856676 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cb703b-9af0-479e-819e-5c7c86b81fd3 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:14.856676 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cb703b-9af0-479e-819e-5c7c86b81fd3 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:14.856676 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cb703b-9af0-479e-819e-5c7c86b81fd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:14.866111 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1547b05a-bd13-4f1a-9b70-202c13741c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:14.869371 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1547b05a-bd13-4f1a-9b70-202c13741c79 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:14.869371 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1547b05a-bd13-4f1a-9b70-202c13741c79 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:14.869371 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1547b05a-bd13-4f1a-9b70-202c13741c79 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:14.869371 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1547b05a-bd13-4f1a-9b70-202c13741c79 None admin] RBAC: Adding query filter params (role.id=43106c664e1b40de984c0c62701149c8, scope.project.id=dc9aa6385a6f44c6b86ffe510a6bc114, user.id=281b2f886b53468d808994f33b5f60ed) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:14.869371 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1547b05a-bd13-4f1a-9b70-202c13741c79 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:14.871461 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1547b05a-bd13-4f1a-9b70-202c13741c79 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:14.878340 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-80cb703b-9af0-479e-819e-5c7c86b81fd3 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:44:14.878340 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-80cb703b-9af0-479e-819e-5c7c86b81fd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:44:14.878575 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 92/182] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Jun 10 12:44:14 2022] GET /identity/v3/role_assignments?user.id=281b2f886b53468d808994f33b5f60ed&scope.project.id=dc9aa6385a6f44c6b86ffe510a6bc114&role.id=43106c664e1b40de984c0c62701149c8 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:14.879965 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1547b05a-bd13-4f1a-9b70-202c13741c79 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:14.939128 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:15.025518 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 91/183] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:44:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 10 12:44:15.026496 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cb703b-9af0-479e-819e-5c7c86b81fd3 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:15.060331 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1547b05a-bd13-4f1a-9b70-202c13741c79 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:15.106403 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941cec6e-27d4-43c9-865c-5c6a975d4a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:15.107554 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941cec6e-27d4-43c9-865c-5c6a975d4a26 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:15.108085 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941cec6e-27d4-43c9-865c-5c6a975d4a26 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:15.108187 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941cec6e-27d4-43c9-865c-5c6a975d4a26 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:15.118709 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941cec6e-27d4-43c9-865c-5c6a975d4a26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:44:15.127875 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-941cec6e-27d4-43c9-865c-5c6a975d4a26 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 10 12:44:15.127875 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:44:15.127875 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:44:15.127875 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:44:15.127875 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:44:15.127875 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:44:15.127875 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:44:15.127875 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:44:15.127875 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:44:15.127875 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:44:15.127875 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:44:15.127875 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:44:15.127875 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:44:15.127875 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:44:15.127875 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:44:15.127875 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:44:15.128785 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:44:15.128785 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:44:15.128785 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:44:15.128785 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:44:15.128785 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:44:15.128785 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:44:15.128785 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:44:15.128785 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:44:15.128785 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:44:15.128785 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:44:15.128785 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:44:15.128785 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:44:15.128785 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:44:15.128785 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:44:15.128785 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:44:15.128785 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:44:15.129578 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:44:15.129578 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:44:15.129578 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:44:15.129578 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:44:15.129578 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:44:15.129578 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:44:15.129578 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:44:15.129578 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:44:15.129578 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:44:15.129578 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:44:15.129578 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 10 12:44:15.129578 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:44:15.129578 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 93/184] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Jun 10 12:44:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:44:15.130742 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04722f10-a2cf-49a6-b414-ad25a5bf9f4d None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:15.130742 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04722f10-a2cf-49a6-b414-ad25a5bf9f4d None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:15.130906 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04722f10-a2cf-49a6-b414-ad25a5bf9f4d None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:15.132900 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 94/185] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:44:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:44:15.132900 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04722f10-a2cf-49a6-b414-ad25a5bf9f4d None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:15.166411 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:15.176520 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cb703b-9af0-479e-819e-5c7c86b81fd3 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:15.215727 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c419e181-2f55-4f00-8680-525f86aa4fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:15.216327 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c419e181-2f55-4f00-8680-525f86aa4fa8 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:15.216616 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c419e181-2f55-4f00-8680-525f86aa4fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:15.216896 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c419e181-2f55-4f00-8680-525f86aa4fa8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:15.217798 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c419e181-2f55-4f00-8680-525f86aa4fa8 None admin] RBAC: Adding query filter params (role.id=2d71ee7c2a544fe5ad77d92b24280efe, scope.project.id=3663edc029e8431d904a228a9e5755a0, user.id=281b2f886b53468d808994f33b5f60ed) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:15.218110 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c419e181-2f55-4f00-8680-525f86aa4fa8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:15.219897 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c419e181-2f55-4f00-8680-525f86aa4fa8 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:15.225594 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 92/186] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Jun 10 12:44:15 2022] GET /identity/v3/role_assignments?user.id=281b2f886b53468d808994f33b5f60ed&scope.project.id=3663edc029e8431d904a228a9e5755a0&role.id=2d71ee7c2a544fe5ad77d92b24280efe => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:15.226191 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c419e181-2f55-4f00-8680-525f86aa4fa8 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:15.243576 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:15.261793 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04722f10-a2cf-49a6-b414-ad25a5bf9f4d None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:15.300157 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ea4478-a27e-473f-ad3e-a9c4614be0a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:15.300991 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ea4478-a27e-473f-ad3e-a9c4614be0a3 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:15.301278 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ea4478-a27e-473f-ad3e-a9c4614be0a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:15.301555 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ea4478-a27e-473f-ad3e-a9c4614be0a3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:15.313877 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ea4478-a27e-473f-ad3e-a9c4614be0a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:44:15.322099 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-d1ea4478-a27e-473f-ad3e-a9c4614be0a3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 10 12:44:15.322099 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:44:15.322099 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:44:15.322099 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:44:15.322099 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:44:15.322099 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:44:15.322099 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:44:15.322099 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:44:15.322099 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:44:15.322099 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:44:15.322099 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:44:15.322099 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:44:15.322099 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:44:15.322099 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:44:15.322099 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:44:15.322099 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:44:15.323123 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:44:15.323123 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:44:15.323123 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:44:15.323123 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:44:15.323123 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:44:15.323123 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:44:15.323123 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:44:15.323123 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:44:15.323123 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:44:15.323123 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:44:15.323123 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:44:15.323123 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:44:15.323123 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:44:15.323123 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:44:15.323123 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:44:15.323123 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:44:15.324358 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:44:15.324358 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:44:15.324358 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:44:15.324358 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:44:15.324358 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:44:15.324358 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:44:15.324358 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:44:15.324358 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:44:15.324358 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:44:15.324358 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:44:15.324358 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 10 12:44:15.324358 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:44:15.324358 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 95/187] 10.222.0.27 () {56 vars in 1114 bytes} [Fri Jun 10 12:44:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:44:15.325316 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36ca9d1-0041-4142-a29e-569aa8a2f655 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:15.325316 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36ca9d1-0041-4142-a29e-569aa8a2f655 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:15.325316 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36ca9d1-0041-4142-a29e-569aa8a2f655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:15.343648 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-b36ca9d1-0041-4142-a29e-569aa8a2f655 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:44:15.344911 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b36ca9d1-0041-4142-a29e-569aa8a2f655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:44:15.345995 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c419e181-2f55-4f00-8680-525f86aa4fa8 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:15.385913 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c72ca6-3339-4fb2-85a2-eea28bb49559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:15.387898 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c72ca6-3339-4fb2-85a2-eea28bb49559 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:15.387898 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c72ca6-3339-4fb2-85a2-eea28bb49559 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:15.387898 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c72ca6-3339-4fb2-85a2-eea28bb49559 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:15.388182 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c72ca6-3339-4fb2-85a2-eea28bb49559 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:15.388350 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c72ca6-3339-4fb2-85a2-eea28bb49559 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:15.389737 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c72ca6-3339-4fb2-85a2-eea28bb49559 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:15.401400 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 93/188] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Jun 10 12:44:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:15.406303 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f11a90-aa89-4c09-a162-7ee8c51ceb7b None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:15.406588 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f11a90-aa89-4c09-a162-7ee8c51ceb7b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:15.406872 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f11a90-aa89-4c09-a162-7ee8c51ceb7b None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:15.408386 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 94/189] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:44:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:44:15.408943 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f11a90-aa89-4c09-a162-7ee8c51ceb7b None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:15.431082 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 96/190] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:44:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 10 12:44:15.432550 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36ca9d1-0041-4142-a29e-569aa8a2f655 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:15.438753 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:15.512945 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:15.527998 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f11a90-aa89-4c09-a162-7ee8c51ceb7b None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:15.534891 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36ca9d1-0041-4142-a29e-569aa8a2f655 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:15.561383 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a76b866-c022-4b77-83a6-6986d5e21f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:15.562050 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a76b866-c022-4b77-83a6-6986d5e21f9b None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:15.562329 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a76b866-c022-4b77-83a6-6986d5e21f9b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:15.562625 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a76b866-c022-4b77-83a6-6986d5e21f9b None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:15.563366 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a76b866-c022-4b77-83a6-6986d5e21f9b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:15.563670 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a76b866-c022-4b77-83a6-6986d5e21f9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:15.565148 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a76b866-c022-4b77-83a6-6986d5e21f9b None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:15.569265 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f52d6e6-ca7b-4d1c-9cbc-999479b8fc72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:15.570134 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f52d6e6-ca7b-4d1c-9cbc-999479b8fc72 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:15.570416 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f52d6e6-ca7b-4d1c-9cbc-999479b8fc72 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:15.570693 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f52d6e6-ca7b-4d1c-9cbc-999479b8fc72 None admin] PATH_INFO: `/v3/users/7576e331fed7406ab308506ab4e67edb` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:15.572883 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 95/191] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Jun 10 12:44:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:15.574127 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef04fb9-0a7c-400a-b0b4-1c43ba3c9a57 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:15.574418 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef04fb9-0a7c-400a-b0b4-1c43ba3c9a57 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:15.574702 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef04fb9-0a7c-400a-b0b4-1c43ba3c9a57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:15.588239 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f52d6e6-ca7b-4d1c-9cbc-999479b8fc72 None admin] RBAC: Authorizing `identity:get_user(user_id=7576e331fed7406ab308506ab4e67edb)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:15.590211 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f52d6e6-ca7b-4d1c-9cbc-999479b8fc72 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:15.590752 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-3ef04fb9-0a7c-400a-b0b4-1c43ba3c9a57 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:44:15.591930 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 97/192] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Jun 10 12:44:15 2022] GET /identity/v3/users/7576e331fed7406ab308506ab4e67edb => generated 283 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:15.592030 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3ef04fb9-0a7c-400a-b0b4-1c43ba3c9a57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:44:15.596757 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f52d6e6-ca7b-4d1c-9cbc-999479b8fc72 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:15.671894 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 96/193] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:44:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 10 12:44:15.671894 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef04fb9-0a7c-400a-b0b4-1c43ba3c9a57 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:15.673290 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:15.694908 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f52d6e6-ca7b-4d1c-9cbc-999479b8fc72 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:15.727527 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42fea20-da2c-4053-aa09-b3233867e0e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:15.728335 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42fea20-da2c-4053-aa09-b3233867e0e5 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:15.728618 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42fea20-da2c-4053-aa09-b3233867e0e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:15.728890 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42fea20-da2c-4053-aa09-b3233867e0e5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:15.731276 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:15.740132 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42fea20-da2c-4053-aa09-b3233867e0e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:44:15.748344 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-b42fea20-da2c-4053-aa09-b3233867e0e5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 10 12:44:15.748344 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:44:15.748344 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:44:15.748344 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:44:15.748344 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:44:15.748344 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:44:15.748344 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:44:15.748344 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:44:15.748344 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:44:15.748344 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:44:15.748344 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:44:15.748344 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:44:15.748344 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:44:15.748344 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:44:15.748344 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:44:15.748344 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:44:15.749253 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:44:15.749253 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:44:15.749253 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:44:15.749253 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:44:15.749253 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:44:15.749253 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:44:15.749253 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:44:15.749253 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:44:15.749253 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:44:15.749253 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:44:15.749253 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:44:15.749253 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:44:15.749253 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:44:15.749253 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:44:15.749253 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:44:15.749253 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:44:15.750144 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:44:15.750144 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:44:15.750144 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:44:15.750144 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:44:15.750144 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:44:15.750144 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:44:15.750144 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:44:15.750144 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:44:15.750144 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:44:15.750144 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:44:15.750144 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 10 12:44:15.750144 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:44:15.750144 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 98/194] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Jun 10 12:44:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:44:15.750837 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42fea20-da2c-4053-aa09-b3233867e0e5 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:15.760766 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef04fb9-0a7c-400a-b0b4-1c43ba3c9a57 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:15.795060 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550516c2-0d68-4ccc-91e8-1ddc56971ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:15.795958 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550516c2-0d68-4ccc-91e8-1ddc56971ad7 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:15.796248 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550516c2-0d68-4ccc-91e8-1ddc56971ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:15.796529 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550516c2-0d68-4ccc-91e8-1ddc56971ad7 None admin] PATH_INFO: `/v3/users/7576e331fed7406ab308506ab4e67edb` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:15.797720 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550516c2-0d68-4ccc-91e8-1ddc56971ad7 None admin] RBAC: Authorizing `identity:get_user(user_id=7576e331fed7406ab308506ab4e67edb)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:15.799185 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550516c2-0d68-4ccc-91e8-1ddc56971ad7 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:15.800844 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 97/195] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Jun 10 12:44:15 2022] GET /identity/v3/users/7576e331fed7406ab308506ab4e67edb => generated 283 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:15.801596 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550516c2-0d68-4ccc-91e8-1ddc56971ad7 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:15.854406 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42fea20-da2c-4053-aa09-b3233867e0e5 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:15.885960 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:15.887493 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43949d0-72fc-43ec-8ce2-33b5b681f493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:15.888166 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43949d0-72fc-43ec-8ce2-33b5b681f493 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:15.888447 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43949d0-72fc-43ec-8ce2-33b5b681f493 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:15.888718 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43949d0-72fc-43ec-8ce2-33b5b681f493 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:15.906104 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550516c2-0d68-4ccc-91e8-1ddc56971ad7 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:15.939055 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a376519d-6c82-4fba-9ddf-d208acc25a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:15.939844 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a376519d-6c82-4fba-9ddf-d208acc25a60 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:15.940161 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a376519d-6c82-4fba-9ddf-d208acc25a60 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:15.940443 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a376519d-6c82-4fba-9ddf-d208acc25a60 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:15.941621 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a376519d-6c82-4fba-9ddf-d208acc25a60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:15.943056 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a376519d-6c82-4fba-9ddf-d208acc25a60 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:15.944709 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 98/196] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:44:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:15.945461 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a376519d-6c82-4fba-9ddf-d208acc25a60 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:15.950949 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:15.958282 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43949d0-72fc-43ec-8ce2-33b5b681f493 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:15.960684 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43949d0-72fc-43ec-8ce2-33b5b681f493 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:16.043199 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a376519d-6c82-4fba-9ddf-d208acc25a60 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:16.057089 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 99/197] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Jun 10 12:44:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 10 12:44:16.061066 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43949d0-72fc-43ec-8ce2-33b5b681f493 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:16.076725 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eecc9bd-535c-4b44-8dfa-5e32fd02066c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:16.077385 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eecc9bd-535c-4b44-8dfa-5e32fd02066c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:16.077678 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eecc9bd-535c-4b44-8dfa-5e32fd02066c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:16.077959 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eecc9bd-535c-4b44-8dfa-5e32fd02066c None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:16.078741 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eecc9bd-535c-4b44-8dfa-5e32fd02066c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:16.079050 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eecc9bd-535c-4b44-8dfa-5e32fd02066c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:16.080464 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eecc9bd-535c-4b44-8dfa-5e32fd02066c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:16.089093 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 99/198] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Jun 10 12:44:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:16.090098 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eecc9bd-535c-4b44-8dfa-5e32fd02066c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:16.102685 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:16.152595 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:16.158861 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43949d0-72fc-43ec-8ce2-33b5b681f493 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:16.189353 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eecc9bd-535c-4b44-8dfa-5e32fd02066c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:16.193552 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24387e4d-9153-4da9-a26a-04ce9a2a2215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:16.194194 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24387e4d-9153-4da9-a26a-04ce9a2a2215 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:16.194475 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24387e4d-9153-4da9-a26a-04ce9a2a2215 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:16.194746 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24387e4d-9153-4da9-a26a-04ce9a2a2215 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:16.221703 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8288c7-f10b-49ef-8220-90df98c667c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:16.222389 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8288c7-f10b-49ef-8220-90df98c667c6 None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:16.222673 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8288c7-f10b-49ef-8220-90df98c667c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:16.222953 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8288c7-f10b-49ef-8220-90df98c667c6 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:16.223660 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8288c7-f10b-49ef-8220-90df98c667c6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:16.225140 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8288c7-f10b-49ef-8220-90df98c667c6 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:16.260871 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24387e4d-9153-4da9-a26a-04ce9a2a2215 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:16.262315 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24387e4d-9153-4da9-a26a-04ce9a2a2215 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:16.299846 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 100/199] 10.222.0.27 () {60 vars in 1132 bytes} [Fri Jun 10 12:44:16 2022] POST /identity/v3/users => generated 297 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 12:44:16.300676 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8288c7-f10b-49ef-8220-90df98c667c6 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:16.364323 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:16.372500 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 100/200] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Jun 10 12:44:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 10 12:44:16.373511 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24387e4d-9153-4da9-a26a-04ce9a2a2215 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:16.382026 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:16.421672 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8288c7-f10b-49ef-8220-90df98c667c6 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:16.458833 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7510488f-138b-4172-83c9-e1c0f439bd1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:16.459702 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7510488f-138b-4172-83c9-e1c0f439bd1f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:16.460015 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7510488f-138b-4172-83c9-e1c0f439bd1f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:16.460279 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7510488f-138b-4172-83c9-e1c0f439bd1f None admin] PATH_INFO: `/v3/projects/485e375bd3394086be6c5cd49bd19e2d` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:16.461502 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7510488f-138b-4172-83c9-e1c0f439bd1f None admin] RBAC: Authorizing `identity:get_project(project_id=485e375bd3394086be6c5cd49bd19e2d)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:16.462965 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7510488f-138b-4172-83c9-e1c0f439bd1f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:16.465268 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 101/201] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Jun 10 12:44:16 2022] GET /identity/v3/projects/485e375bd3394086be6c5cd49bd19e2d => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:16.466110 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7510488f-138b-4172-83c9-e1c0f439bd1f None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:16.491919 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24387e4d-9153-4da9-a26a-04ce9a2a2215 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:16.534796 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fcfa76-4fea-48fa-9d52-472b40d73773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:16.535894 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fcfa76-4fea-48fa-9d52-472b40d73773 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:16.536128 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fcfa76-4fea-48fa-9d52-472b40d73773 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:16.536401 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fcfa76-4fea-48fa-9d52-472b40d73773 None admin] PATH_INFO: `/v3/users/b1832bf1dd584457ace688490fd59d10` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:16.537872 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fcfa76-4fea-48fa-9d52-472b40d73773 None admin] RBAC: Authorizing `identity:get_user(user_id=b1832bf1dd584457ace688490fd59d10)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:16.539387 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fcfa76-4fea-48fa-9d52-472b40d73773 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:16.541192 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 101/202] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Jun 10 12:44:16 2022] GET /identity/v3/users/b1832bf1dd584457ace688490fd59d10 => generated 289 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:16.543924 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fcfa76-4fea-48fa-9d52-472b40d73773 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:16.591045 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7510488f-138b-4172-83c9-e1c0f439bd1f None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:16.620256 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:16.632113 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cbdb46-1f60-4b5d-81c1-c82a6beabf59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:16.632986 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cbdb46-1f60-4b5d-81c1-c82a6beabf59 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:16.633270 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cbdb46-1f60-4b5d-81c1-c82a6beabf59 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:16.633548 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cbdb46-1f60-4b5d-81c1-c82a6beabf59 None admin] PATH_INFO: `/v3/projects/485e375bd3394086be6c5cd49bd19e2d` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:16.634732 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cbdb46-1f60-4b5d-81c1-c82a6beabf59 None admin] RBAC: Authorizing `identity:get_project(project_id=485e375bd3394086be6c5cd49bd19e2d)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:16.637738 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cbdb46-1f60-4b5d-81c1-c82a6beabf59 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:16.639075 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 102/203] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Jun 10 12:44:16 2022] GET /identity/v3/projects/485e375bd3394086be6c5cd49bd19e2d => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:16.639839 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cbdb46-1f60-4b5d-81c1-c82a6beabf59 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:16.643551 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:16.673485 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fcfa76-4fea-48fa-9d52-472b40d73773 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:16.709863 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0044fd19-0256-4055-97ed-76f13d669eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:16.710551 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0044fd19-0256-4055-97ed-76f13d669eff None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:16.710834 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0044fd19-0256-4055-97ed-76f13d669eff None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:16.711110 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0044fd19-0256-4055-97ed-76f13d669eff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:16.711970 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0044fd19-0256-4055-97ed-76f13d669eff None admin] RBAC: Adding query filter params (role.id=b43d5a8247e244869c2c70f986f42e8a, scope.project.id=485e375bd3394086be6c5cd49bd19e2d, user.id=7576e331fed7406ab308506ab4e67edb) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:16.712285 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0044fd19-0256-4055-97ed-76f13d669eff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:16.714348 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0044fd19-0256-4055-97ed-76f13d669eff None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:16.720058 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 102/204] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Jun 10 12:44:16 2022] GET /identity/v3/role_assignments?user.id=7576e331fed7406ab308506ab4e67edb&scope.project.id=485e375bd3394086be6c5cd49bd19e2d&role.id=b43d5a8247e244869c2c70f986f42e8a => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:16.720825 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0044fd19-0256-4055-97ed-76f13d669eff None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:16.795111 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cbdb46-1f60-4b5d-81c1-c82a6beabf59 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:16.883118 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9280d807-6653-415a-bda1-50cdeed5497b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:16.884954 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9280d807-6653-415a-bda1-50cdeed5497b None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:16.885307 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9280d807-6653-415a-bda1-50cdeed5497b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:16.886093 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9280d807-6653-415a-bda1-50cdeed5497b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:16.929990 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0044fd19-0256-4055-97ed-76f13d669eff None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:16.961063 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:16.973124 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9280d807-6653-415a-bda1-50cdeed5497b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:16.974716 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9280d807-6653-415a-bda1-50cdeed5497b None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:16.984562 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbba9e07-5a89-42cf-a396-0a554372d492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:16.985478 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbba9e07-5a89-42cf-a396-0a554372d492 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:16.985965 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbba9e07-5a89-42cf-a396-0a554372d492 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:16.986109 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:16.986269 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbba9e07-5a89-42cf-a396-0a554372d492 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:16.987083 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbba9e07-5a89-42cf-a396-0a554372d492 None admin] RBAC: Adding query filter params (role.id=2d71ee7c2a544fe5ad77d92b24280efe, scope.project.id=485e375bd3394086be6c5cd49bd19e2d, user.id=7576e331fed7406ab308506ab4e67edb) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:16.987394 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbba9e07-5a89-42cf-a396-0a554372d492 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:16.988869 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbba9e07-5a89-42cf-a396-0a554372d492 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:16.995727 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 103/205] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Jun 10 12:44:16 2022] GET /identity/v3/role_assignments?user.id=7576e331fed7406ab308506ab4e67edb&scope.project.id=485e375bd3394086be6c5cd49bd19e2d&role.id=2d71ee7c2a544fe5ad77d92b24280efe => generated 259 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:17.093263 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 103/206] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Jun 10 12:44:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 10 12:44:17.099713 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbba9e07-5a89-42cf-a396-0a554372d492 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:17.332226 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbba9e07-5a89-42cf-a396-0a554372d492 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:17.373940 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae79dcb-61e7-4a30-b060-4f06613a76c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:17.375111 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae79dcb-61e7-4a30-b060-4f06613a76c5 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:17.375616 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae79dcb-61e7-4a30-b060-4f06613a76c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:17.376164 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae79dcb-61e7-4a30-b060-4f06613a76c5 None admin] PATH_INFO: `/v3/projects/3663edc029e8431d904a228a9e5755a0` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:17.377559 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae79dcb-61e7-4a30-b060-4f06613a76c5 None admin] RBAC: Authorizing `identity:get_project(project_id=3663edc029e8431d904a228a9e5755a0)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:17.379185 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae79dcb-61e7-4a30-b060-4f06613a76c5 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:17.381104 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 104/207] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Jun 10 12:44:17 2022] GET /identity/v3/projects/3663edc029e8431d904a228a9e5755a0 => generated 298 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:17.388953 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9280d807-6653-415a-bda1-50cdeed5497b None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:17.521729 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9280d807-6653-415a-bda1-50cdeed5497b None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:17.523316 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:17.559430 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea195d6-bae3-4719-b072-ce21e6243311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:17.560109 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea195d6-bae3-4719-b072-ce21e6243311 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:17.560393 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea195d6-bae3-4719-b072-ce21e6243311 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:17.560668 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea195d6-bae3-4719-b072-ce21e6243311 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:17.561503 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea195d6-bae3-4719-b072-ce21e6243311 None admin] RBAC: Adding query filter params (role.id=7d6c4b2adb9d43bca1a6573e8f7a9381, scope.project.id=3663edc029e8431d904a228a9e5755a0, user.id=b1832bf1dd584457ace688490fd59d10) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:17.561815 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea195d6-bae3-4719-b072-ce21e6243311 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:17.563236 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea195d6-bae3-4719-b072-ce21e6243311 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:17.569307 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 104/208] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Jun 10 12:44:17 2022] GET /identity/v3/role_assignments?user.id=b1832bf1dd584457ace688490fd59d10&scope.project.id=3663edc029e8431d904a228a9e5755a0&role.id=7d6c4b2adb9d43bca1a6573e8f7a9381 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:18.744603 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1737746f-8cdc-490f-addf-2a631f699c5c None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:18.745119 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1737746f-8cdc-490f-addf-2a631f699c5c None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:18.745194 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1737746f-8cdc-490f-addf-2a631f699c5c None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:18.746769 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 105/209] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:44:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:44:18.752124 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ef3d28-18c7-4b34-9309-0fe381333cd0 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:18.752519 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ef3d28-18c7-4b34-9309-0fe381333cd0 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:18.752685 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ef3d28-18c7-4b34-9309-0fe381333cd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:18.770164 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-a5ef3d28-18c7-4b34-9309-0fe381333cd0 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:44:18.771373 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a5ef3d28-18c7-4b34-9309-0fe381333cd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:44:18.847762 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 105/210] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:44:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 10 12:44:18.860587 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc46127-5566-4e30-bb1f-a02cd85108f7 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:18.861184 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc46127-5566-4e30-bb1f-a02cd85108f7 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:18.861775 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc46127-5566-4e30-bb1f-a02cd85108f7 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:18.866531 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 106/211] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:44:18 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:44:18.874178 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd14dba8-9d5c-43de-aaae-17227d925e01 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:18.874477 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd14dba8-9d5c-43de-aaae-17227d925e01 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:18.874763 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd14dba8-9d5c-43de-aaae-17227d925e01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:18.875575 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ef3d28-18c7-4b34-9309-0fe381333cd0 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:18.893591 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-bd14dba8-9d5c-43de-aaae-17227d925e01 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:44:18.895306 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-bd14dba8-9d5c-43de-aaae-17227d925e01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:44:18.953117 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:18.968074 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 107/212] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:44:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 10 12:44:18.980640 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:18.986326 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ef3d28-18c7-4b34-9309-0fe381333cd0 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:18.993704 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd14dba8-9d5c-43de-aaae-17227d925e01 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:19.023034 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c34a06a-dab9-44ea-bd81-2ed9c1101178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:19.023765 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c34a06a-dab9-44ea-bd81-2ed9c1101178 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:19.024074 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c34a06a-dab9-44ea-bd81-2ed9c1101178 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:19.024353 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c34a06a-dab9-44ea-bd81-2ed9c1101178 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:19.036082 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c34a06a-dab9-44ea-bd81-2ed9c1101178 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:44:19.045236 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-9c34a06a-dab9-44ea-bd81-2ed9c1101178 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 10 12:44:19.045236 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:44:19.045236 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:44:19.045236 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:44:19.045236 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:44:19.045236 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:44:19.045236 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:44:19.045236 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:44:19.045236 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:44:19.045236 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:44:19.045236 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:44:19.045236 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:44:19.045236 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:44:19.045236 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:44:19.045236 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:44:19.045236 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:44:19.046158 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:44:19.046158 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:44:19.046158 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:44:19.046158 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:44:19.046158 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:44:19.046158 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:44:19.046158 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:44:19.046158 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:44:19.046158 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:44:19.046158 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:44:19.046158 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:44:19.046158 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:44:19.046158 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:44:19.046158 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:44:19.046158 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:44:19.046158 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:44:19.046960 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:44:19.046960 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:44:19.046960 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:44:19.046960 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:44:19.046960 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:44:19.046960 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:44:19.046960 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:44:19.046960 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:44:19.046960 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:44:19.046960 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:44:19.046960 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 10 12:44:19.046960 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:44:19.046960 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 106/213] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Jun 10 12:44:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:44:19.052448 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c34a06a-dab9-44ea-bd81-2ed9c1101178 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:19.104282 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd14dba8-9d5c-43de-aaae-17227d925e01 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:19.141531 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528da708-5c63-48e7-9d26-10f9de052e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:19.142762 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528da708-5c63-48e7-9d26-10f9de052e5f None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:19.143240 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528da708-5c63-48e7-9d26-10f9de052e5f None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:19.143771 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528da708-5c63-48e7-9d26-10f9de052e5f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:19.145164 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528da708-5c63-48e7-9d26-10f9de052e5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:19.147026 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528da708-5c63-48e7-9d26-10f9de052e5f None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:19.148871 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 108/214] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:44:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 12:44:19.154565 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528da708-5c63-48e7-9d26-10f9de052e5f None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:19.163474 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c34a06a-dab9-44ea-bd81-2ed9c1101178 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:19.206357 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936dd8bc-23fb-4550-86ec-78ea62e2b541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:19.207066 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936dd8bc-23fb-4550-86ec-78ea62e2b541 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:19.207357 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936dd8bc-23fb-4550-86ec-78ea62e2b541 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:19.207673 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936dd8bc-23fb-4550-86ec-78ea62e2b541 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:19.208740 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936dd8bc-23fb-4550-86ec-78ea62e2b541 None admin] RBAC: Adding query filter params (name=member) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:19.209055 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936dd8bc-23fb-4550-86ec-78ea62e2b541 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:19.210575 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936dd8bc-23fb-4550-86ec-78ea62e2b541 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:19.220904 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 107/215] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Jun 10 12:44:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 12:44:19.225936 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:19.227050 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936dd8bc-23fb-4550-86ec-78ea62e2b541 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:19.253448 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:19.274797 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528da708-5c63-48e7-9d26-10f9de052e5f None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:19.308031 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8cc247-70a8-454c-b3e2-efd85f770bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:19.308692 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8cc247-70a8-454c-b3e2-efd85f770bdd None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:19.308975 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8cc247-70a8-454c-b3e2-efd85f770bdd None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:19.309247 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8cc247-70a8-454c-b3e2-efd85f770bdd None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:19.309947 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8cc247-70a8-454c-b3e2-efd85f770bdd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:19.311398 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8cc247-70a8-454c-b3e2-efd85f770bdd None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:19.334546 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936dd8bc-23fb-4550-86ec-78ea62e2b541 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:19.367032 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342d1353-bb00-4bf4-bd49-3451018962de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:19.367958 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342d1353-bb00-4bf4-bd49-3451018962de None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:19.368244 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342d1353-bb00-4bf4-bd49-3451018962de None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:19.368517 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342d1353-bb00-4bf4-bd49-3451018962de None admin] PATH_INFO: `/v3/users/62cb9763157a443ab6b016f817628b2e` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:19.381990 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342d1353-bb00-4bf4-bd49-3451018962de None admin] RBAC: Authorizing `identity:get_user(user_id=62cb9763157a443ab6b016f817628b2e)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:19.383415 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342d1353-bb00-4bf4-bd49-3451018962de None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:19.385147 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 108/216] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Jun 10 12:44:19 2022] GET /identity/v3/users/62cb9763157a443ab6b016f817628b2e => generated 297 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 12:44:19.386948 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0916598-7b94-4480-8005-542c45a546aa None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:19.387230 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0916598-7b94-4480-8005-542c45a546aa None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:19.387507 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0916598-7b94-4480-8005-542c45a546aa None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:19.388748 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 109/217] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:44:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:44:19.388748 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2508a011-dff8-4e9d-96e5-52ed4f7bf032 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:19.390353 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2508a011-dff8-4e9d-96e5-52ed4f7bf032 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:19.390512 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2508a011-dff8-4e9d-96e5-52ed4f7bf032 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:19.391712 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 110/218] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:44:19 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:44:19.393322 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2508a011-dff8-4e9d-96e5-52ed4f7bf032 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:19.408896 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 109/219] 10.222.0.27 () {60 vars in 1132 bytes} [Fri Jun 10 12:44:19 2022] POST /identity/v3/users => generated 297 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 10 12:44:19.410331 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8039330-af74-4188-928a-5c22d280b0e7 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:19.410629 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8039330-af74-4188-928a-5c22d280b0e7 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:19.410909 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8039330-af74-4188-928a-5c22d280b0e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:19.430846 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-c8039330-af74-4188-928a-5c22d280b0e7 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:44:19.432675 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c8039330-af74-4188-928a-5c22d280b0e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:44:19.485595 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:19.508329 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 110/220] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:44:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 10 12:44:19.509650 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f15e85-dec0-4e33-84ac-873f4d39c86d None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:19.509936 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f15e85-dec0-4e33-84ac-873f4d39c86d None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:19.510315 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f15e85-dec0-4e33-84ac-873f4d39c86d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:19.522941 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2508a011-dff8-4e9d-96e5-52ed4f7bf032 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:19.529195 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-33f15e85-dec0-4e33-84ac-873f4d39c86d None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:44:19.530651 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-33f15e85-dec0-4e33-84ac-873f4d39c86d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:44:19.563947 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e6ecf8-63cc-433c-9002-18eb30300b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:19.564856 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e6ecf8-63cc-433c-9002-18eb30300b49 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:19.565143 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e6ecf8-63cc-433c-9002-18eb30300b49 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:19.565635 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e6ecf8-63cc-433c-9002-18eb30300b49 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:19.568840 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:19.606972 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 111/221] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:44:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 10 12:44:19.608192 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f15e85-dec0-4e33-84ac-873f4d39c86d None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:19.645241 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e6ecf8-63cc-433c-9002-18eb30300b49 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:19.646704 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e6ecf8-63cc-433c-9002-18eb30300b49 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:19.730827 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f15e85-dec0-4e33-84ac-873f4d39c86d None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:19.747529 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:19.769363 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31729cb6-269f-4387-85c6-6ef280546351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:19.770185 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31729cb6-269f-4387-85c6-6ef280546351 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:19.770464 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31729cb6-269f-4387-85c6-6ef280546351 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:19.770914 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31729cb6-269f-4387-85c6-6ef280546351 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:19.777588 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 111/222] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Jun 10 12:44:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 10 12:44:19.778422 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e6ecf8-63cc-433c-9002-18eb30300b49 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:19.784395 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31729cb6-269f-4387-85c6-6ef280546351 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:44:19.795073 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-31729cb6-269f-4387-85c6-6ef280546351 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 10 12:44:19.795073 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:44:19.795073 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:44:19.795073 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:44:19.795073 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:44:19.795073 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:44:19.795073 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:44:19.795073 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:44:19.795073 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:44:19.795073 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:44:19.795073 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:44:19.795073 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:44:19.795073 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:44:19.795073 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:44:19.795073 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:44:19.795073 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:44:19.796039 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:44:19.796039 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:44:19.796039 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:44:19.796039 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:44:19.796039 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:44:19.796039 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:44:19.796039 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:44:19.796039 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:44:19.796039 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:44:19.796039 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:44:19.796039 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:44:19.796039 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:44:19.796039 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:44:19.796039 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:44:19.796039 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:44:19.796039 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:44:19.797015 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:44:19.797015 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:44:19.797015 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:44:19.797015 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:44:19.797015 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:44:19.797015 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:44:19.797015 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:44:19.797015 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:44:19.797015 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:44:19.797015 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:44:19.797015 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 10 12:44:19.797015 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:44:19.797015 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 112/223] 10.222.0.27 () {56 vars in 1114 bytes} [Fri Jun 10 12:44:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:44:19.797833 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31729cb6-269f-4387-85c6-6ef280546351 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:19.830043 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:19.911269 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e6ecf8-63cc-433c-9002-18eb30300b49 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:19.924238 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31729cb6-269f-4387-85c6-6ef280546351 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:19.947953 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17ca708-946d-4495-8e0e-fb50511e9fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:19.949118 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ca708-946d-4495-8e0e-fb50511e9fcf None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:19.949599 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ca708-946d-4495-8e0e-fb50511e9fcf None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:19.950080 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ca708-946d-4495-8e0e-fb50511e9fcf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:19.960995 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17ca708-946d-4495-8e0e-fb50511e9fcf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:44:19.966007 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a426b7-8cd3-4f6a-b4b6-6f3e9b8e079b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:19.966878 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a426b7-8cd3-4f6a-b4b6-6f3e9b8e079b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:19.967167 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a426b7-8cd3-4f6a-b4b6-6f3e9b8e079b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:19.967533 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a426b7-8cd3-4f6a-b4b6-6f3e9b8e079b None admin] PATH_INFO: `/v3/projects/485e375bd3394086be6c5cd49bd19e2d` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:19.968871 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a426b7-8cd3-4f6a-b4b6-6f3e9b8e079b None admin] RBAC: Authorizing `identity:get_project(project_id=485e375bd3394086be6c5cd49bd19e2d)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:19.970399 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a426b7-8cd3-4f6a-b4b6-6f3e9b8e079b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:19.971247 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-f17ca708-946d-4495-8e0e-fb50511e9fcf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 10 12:44:19.971247 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:44:19.971247 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:44:19.971247 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:44:19.971247 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:44:19.971247 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:44:19.971247 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:44:19.971247 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:44:19.971247 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:44:19.971247 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:44:19.971247 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:44:19.971247 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:44:19.971247 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:44:19.971247 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:44:19.971247 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:44:19.971247 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:44:19.972165 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:44:19.972165 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:44:19.972165 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:44:19.972165 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:44:19.972165 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:44:19.972165 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:44:19.972165 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:44:19.972165 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:44:19.972165 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:44:19.972165 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:44:19.972165 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:44:19.972165 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:44:19.972165 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:44:19.972165 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:44:19.972165 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:44:19.972165 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:44:19.972973 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:44:19.972973 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:44:19.972973 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:44:19.972973 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:44:19.972973 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:44:19.972973 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:44:19.972973 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:44:19.972973 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:44:19.972973 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:44:19.972973 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:44:19.972973 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 10 12:44:19.972973 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:44:19.973584 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 113/224] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Jun 10 12:44:19 2022] GET /identity/v3/projects/485e375bd3394086be6c5cd49bd19e2d => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:19.973584 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a426b7-8cd3-4f6a-b4b6-6f3e9b8e079b None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:19.980999 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 112/225] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Jun 10 12:44:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:44:19.982077 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17ca708-946d-4495-8e0e-fb50511e9fcf None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:20.020855 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:20.083136 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:20.086489 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a426b7-8cd3-4f6a-b4b6-6f3e9b8e079b None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:20.091438 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17ca708-946d-4495-8e0e-fb50511e9fcf None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:20.127687 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3fcc5b-289e-4dd4-97ad-e9ecac3a52fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:20.128717 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3fcc5b-289e-4dd4-97ad-e9ecac3a52fb None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:20.129205 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3fcc5b-289e-4dd4-97ad-e9ecac3a52fb None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:20.129683 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3fcc5b-289e-4dd4-97ad-e9ecac3a52fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:20.129809 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09fb23e-86f6-4142-a8d1-3cc2e08c6bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:20.130439 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09fb23e-86f6-4142-a8d1-3cc2e08c6bb6 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:20.130796 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09fb23e-86f6-4142-a8d1-3cc2e08c6bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:20.131078 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09fb23e-86f6-4142-a8d1-3cc2e08c6bb6 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:20.131515 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3fcc5b-289e-4dd4-97ad-e9ecac3a52fb None admin] RBAC: Adding query filter params (role.id=43106c664e1b40de984c0c62701149c8, scope.project.id=485e375bd3394086be6c5cd49bd19e2d, user.id=62cb9763157a443ab6b016f817628b2e) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:20.131831 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09fb23e-86f6-4142-a8d1-3cc2e08c6bb6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:20.132179 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09fb23e-86f6-4142-a8d1-3cc2e08c6bb6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:20.132439 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3fcc5b-289e-4dd4-97ad-e9ecac3a52fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:20.133558 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09fb23e-86f6-4142-a8d1-3cc2e08c6bb6 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:20.134319 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3fcc5b-289e-4dd4-97ad-e9ecac3a52fb None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:20.140676 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 113/226] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Jun 10 12:44:19 2022] GET /identity/v3/role_assignments?user.id=62cb9763157a443ab6b016f817628b2e&scope.project.id=485e375bd3394086be6c5cd49bd19e2d&role.id=43106c664e1b40de984c0c62701149c8 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:20.145834 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 114/227] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Jun 10 12:44:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:20.145834 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09fb23e-86f6-4142-a8d1-3cc2e08c6bb6 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:20.152889 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3fcc5b-289e-4dd4-97ad-e9ecac3a52fb None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:20.300211 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09fb23e-86f6-4142-a8d1-3cc2e08c6bb6 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:20.354136 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:20.356788 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3fcc5b-289e-4dd4-97ad-e9ecac3a52fb None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:20.392309 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6694e9fe-6769-4ab1-bfd3-8f96068c7ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:20.392988 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6694e9fe-6769-4ab1-bfd3-8f96068c7ef3 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:20.393269 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6694e9fe-6769-4ab1-bfd3-8f96068c7ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:20.393541 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6694e9fe-6769-4ab1-bfd3-8f96068c7ef3 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:20.394295 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6694e9fe-6769-4ab1-bfd3-8f96068c7ef3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:20.394599 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6694e9fe-6769-4ab1-bfd3-8f96068c7ef3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:20.396007 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6694e9fe-6769-4ab1-bfd3-8f96068c7ef3 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:20.411903 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 115/228] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Jun 10 12:44:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:20.416933 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6694e9fe-6769-4ab1-bfd3-8f96068c7ef3 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:20.428142 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8b4251-3b2b-41c4-a09b-cc42c30404ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:20.429032 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8b4251-3b2b-41c4-a09b-cc42c30404ae None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:20.429309 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8b4251-3b2b-41c4-a09b-cc42c30404ae None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:20.429583 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8b4251-3b2b-41c4-a09b-cc42c30404ae None admin] PATH_INFO: `/v3/users/7576e331fed7406ab308506ab4e67edb` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:20.432781 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8b4251-3b2b-41c4-a09b-cc42c30404ae None admin] RBAC: Authorizing `identity:get_user(user_id=7576e331fed7406ab308506ab4e67edb)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:20.434237 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8b4251-3b2b-41c4-a09b-cc42c30404ae None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:20.437705 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 114/229] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Jun 10 12:44:20 2022] GET /identity/v3/users/7576e331fed7406ab308506ab4e67edb => generated 283 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 12:44:20.443804 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8b4251-3b2b-41c4-a09b-cc42c30404ae None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:20.468156 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:20.593815 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6694e9fe-6769-4ab1-bfd3-8f96068c7ef3 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:20.629739 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565f08a1-a21c-4e44-8fbc-d6055e0b837d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:20.630631 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565f08a1-a21c-4e44-8fbc-d6055e0b837d None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:20.630912 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565f08a1-a21c-4e44-8fbc-d6055e0b837d None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:20.631185 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565f08a1-a21c-4e44-8fbc-d6055e0b837d None admin] PATH_INFO: `/v3/users/7576e331fed7406ab308506ab4e67edb` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:20.632532 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565f08a1-a21c-4e44-8fbc-d6055e0b837d None admin] RBAC: Authorizing `identity:get_user(user_id=7576e331fed7406ab308506ab4e67edb)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:20.634030 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565f08a1-a21c-4e44-8fbc-d6055e0b837d None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:20.636397 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 116/230] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Jun 10 12:44:20 2022] GET /identity/v3/users/7576e331fed7406ab308506ab4e67edb => generated 283 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 12:44:20.641991 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565f08a1-a21c-4e44-8fbc-d6055e0b837d None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:20.676754 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8b4251-3b2b-41c4-a09b-cc42c30404ae None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:20.737159 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:20.746216 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565f08a1-a21c-4e44-8fbc-d6055e0b837d None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:20.759210 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15ad70d-a5a0-4b0a-be86-f38f3a226e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:20.760871 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15ad70d-a5a0-4b0a-be86-f38f3a226e94 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:20.761455 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15ad70d-a5a0-4b0a-be86-f38f3a226e94 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:20.763222 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15ad70d-a5a0-4b0a-be86-f38f3a226e94 None admin] PATH_INFO: `/v3/projects/485e375bd3394086be6c5cd49bd19e2d` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:20.765348 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15ad70d-a5a0-4b0a-be86-f38f3a226e94 None admin] RBAC: Authorizing `identity:get_project(project_id=485e375bd3394086be6c5cd49bd19e2d)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:20.767694 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15ad70d-a5a0-4b0a-be86-f38f3a226e94 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:20.772092 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 115/231] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Jun 10 12:44:20 2022] GET /identity/v3/projects/485e375bd3394086be6c5cd49bd19e2d => generated 302 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:20.780781 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15ad70d-a5a0-4b0a-be86-f38f3a226e94 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:20.793658 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be3b994-f8c8-4302-af82-fb7c0a4cd735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:20.795151 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be3b994-f8c8-4302-af82-fb7c0a4cd735 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:20.795628 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be3b994-f8c8-4302-af82-fb7c0a4cd735 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:20.796134 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be3b994-f8c8-4302-af82-fb7c0a4cd735 None admin] PATH_INFO: `/v3/projects/485e375bd3394086be6c5cd49bd19e2d` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:20.797700 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be3b994-f8c8-4302-af82-fb7c0a4cd735 None admin] RBAC: Authorizing `identity:get_project(project_id=485e375bd3394086be6c5cd49bd19e2d)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:20.799470 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be3b994-f8c8-4302-af82-fb7c0a4cd735 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:20.801460 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 117/232] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Jun 10 12:44:20 2022] GET /identity/v3/projects/485e375bd3394086be6c5cd49bd19e2d => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 12:44:20.807601 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be3b994-f8c8-4302-af82-fb7c0a4cd735 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:20.850497 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:20.920434 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15ad70d-a5a0-4b0a-be86-f38f3a226e94 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:20.955651 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c648e2-f976-40cb-8c24-66374c5e8575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:20.956956 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c648e2-f976-40cb-8c24-66374c5e8575 None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:20.957434 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c648e2-f976-40cb-8c24-66374c5e8575 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:20.957902 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c648e2-f976-40cb-8c24-66374c5e8575 None admin] PATH_INFO: `/v3/projects/485e375bd3394086be6c5cd49bd19e2d/users/7576e331fed7406ab308506ab4e67edb/roles/2d71ee7c2a544fe5ad77d92b24280efe` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:20.960367 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c648e2-f976-40cb-8c24-66374c5e8575 None admin] RBAC: Authorizing `identity:create_grant(project_id=485e375bd3394086be6c5cd49bd19e2d, user_id=7576e331fed7406ab308506ab4e67edb, role_id=2d71ee7c2a544fe5ad77d92b24280efe)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:20.962047 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c648e2-f976-40cb-8c24-66374c5e8575 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:21.026789 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be3b994-f8c8-4302-af82-fb7c0a4cd735 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:21.045426 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 116/233] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Jun 10 12:44:20 2022] PUT /identity/v3/projects/485e375bd3394086be6c5cd49bd19e2d/users/7576e331fed7406ab308506ab4e67edb/roles/2d71ee7c2a544fe5ad77d92b24280efe => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 12:44:21.080379 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da047d04-c5c7-41fd-8e8d-86b5777b4a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:21.081674 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da047d04-c5c7-41fd-8e8d-86b5777b4a43 None admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:21.082180 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da047d04-c5c7-41fd-8e8d-86b5777b4a43 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:21.082647 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da047d04-c5c7-41fd-8e8d-86b5777b4a43 None admin] PATH_INFO: `/v3/projects/485e375bd3394086be6c5cd49bd19e2d/users/7576e331fed7406ab308506ab4e67edb/roles/b43d5a8247e244869c2c70f986f42e8a` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:21.084871 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da047d04-c5c7-41fd-8e8d-86b5777b4a43 None admin] RBAC: Authorizing `identity:create_grant(project_id=485e375bd3394086be6c5cd49bd19e2d, user_id=7576e331fed7406ab308506ab4e67edb, role_id=b43d5a8247e244869c2c70f986f42e8a)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:21.086629 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da047d04-c5c7-41fd-8e8d-86b5777b4a43 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:21.198972 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 118/234] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Jun 10 12:44:20 2022] PUT /identity/v3/projects/485e375bd3394086be6c5cd49bd19e2d/users/7576e331fed7406ab308506ab4e67edb/roles/b43d5a8247e244869c2c70f986f42e8a => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 12:44:21.768229 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3843187-29f1-4a03-869f-ace5ac20782d None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:21.768934 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3843187-29f1-4a03-869f-ace5ac20782d None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:21.769433 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3843187-29f1-4a03-869f-ace5ac20782d None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:21.771253 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 117/235] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:44:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:44:21.776756 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5a1687-7855-4549-a956-99a8def4ecb5 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:21.777257 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5a1687-7855-4549-a956-99a8def4ecb5 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:21.777725 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5a1687-7855-4549-a956-99a8def4ecb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:21.799813 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-1c5a1687-7855-4549-a956-99a8def4ecb5 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:44:21.801764 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-1c5a1687-7855-4549-a956-99a8def4ecb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:44:21.820184 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:21.887348 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 119/236] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:44:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 10 12:44:21.918863 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3843187-29f1-4a03-869f-ace5ac20782d None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:22.044643 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:22.052259 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3843187-29f1-4a03-869f-ace5ac20782d None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:22.058807 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658c819f-34b7-4868-a62c-f2cd1afe5095 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:22.059100 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658c819f-34b7-4868-a62c-f2cd1afe5095 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:22.059643 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658c819f-34b7-4868-a62c-f2cd1afe5095 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:22.060797 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 120/237] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:44:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:44:22.066121 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8086ec-144a-4902-8b21-e19993a99c36 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:22.066519 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8086ec-144a-4902-8b21-e19993a99c36 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:22.066906 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8086ec-144a-4902-8b21-e19993a99c36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:22.086937 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-dd8086ec-144a-4902-8b21-e19993a99c36 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:44:22.089452 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-dd8086ec-144a-4902-8b21-e19993a99c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:44:22.100578 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa55576-b921-4ac9-9764-acb968f2cc48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:22.101395 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa55576-b921-4ac9-9764-acb968f2cc48 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:22.101742 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa55576-b921-4ac9-9764-acb968f2cc48 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:22.101955 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa55576-b921-4ac9-9764-acb968f2cc48 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:22.128112 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa55576-b921-4ac9-9764-acb968f2cc48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:44:22.137900 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-6aa55576-b921-4ac9-9764-acb968f2cc48 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 10 12:44:22.137900 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:44:22.137900 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:44:22.137900 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:44:22.137900 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:44:22.137900 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:44:22.137900 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:44:22.137900 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:44:22.137900 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:44:22.137900 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:44:22.137900 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:44:22.137900 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:44:22.137900 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:44:22.137900 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:44:22.137900 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:44:22.137900 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:44:22.138855 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:44:22.138855 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:44:22.138855 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:44:22.138855 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:44:22.138855 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:44:22.138855 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:44:22.138855 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:44:22.138855 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:44:22.138855 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:44:22.138855 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:44:22.138855 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:44:22.138855 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:44:22.138855 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:44:22.138855 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:44:22.138855 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:44:22.138855 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:44:22.139770 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:44:22.139770 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:44:22.139770 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:44:22.139770 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:44:22.139770 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:44:22.139770 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:44:22.139770 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:44:22.139770 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:44:22.139770 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:44:22.139770 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:44:22.139770 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 10 12:44:22.139770 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:44:22.139770 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 118/238] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Jun 10 12:44:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:44:22.148701 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa55576-b921-4ac9-9764-acb968f2cc48 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:22.199704 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 121/239] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:44:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 10 12:44:22.275053 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8086ec-144a-4902-8b21-e19993a99c36 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:22.308862 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa55576-b921-4ac9-9764-acb968f2cc48 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:22.363175 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa6eed3-3c56-49bb-9da7-85a9b2d97bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:22.363842 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa6eed3-3c56-49bb-9da7-85a9b2d97bf6 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:22.364144 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa6eed3-3c56-49bb-9da7-85a9b2d97bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:22.364419 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa6eed3-3c56-49bb-9da7-85a9b2d97bf6 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:22.365174 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa6eed3-3c56-49bb-9da7-85a9b2d97bf6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:22.365478 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa6eed3-3c56-49bb-9da7-85a9b2d97bf6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:22.366847 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa6eed3-3c56-49bb-9da7-85a9b2d97bf6 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:22.384494 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 119/240] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Jun 10 12:44:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:22.390712 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa6eed3-3c56-49bb-9da7-85a9b2d97bf6 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:22.422526 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:22.428591 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:22.451691 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8086ec-144a-4902-8b21-e19993a99c36 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:22.516910 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fa22d6-ee10-4cef-b745-050941d9c633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:22.517735 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fa22d6-ee10-4cef-b745-050941d9c633 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:22.518013 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fa22d6-ee10-4cef-b745-050941d9c633 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:22.518282 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fa22d6-ee10-4cef-b745-050941d9c633 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:22.520950 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fa22d6-ee10-4cef-b745-050941d9c633 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:22.522446 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fa22d6-ee10-4cef-b745-050941d9c633 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:22.526148 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 122/241] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:44:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 12:44:22.532898 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fa22d6-ee10-4cef-b745-050941d9c633 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:22.577514 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa6eed3-3c56-49bb-9da7-85a9b2d97bf6 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:22.613066 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5cd522-04d9-4310-8a66-d81cd7228089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:22.613941 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5cd522-04d9-4310-8a66-d81cd7228089 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:22.614223 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5cd522-04d9-4310-8a66-d81cd7228089 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:22.614500 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5cd522-04d9-4310-8a66-d81cd7228089 None admin] PATH_INFO: `/v3/users/ef91f0d46059481db5f9a5a0eb635aae` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:22.630324 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5cd522-04d9-4310-8a66-d81cd7228089 None admin] RBAC: Authorizing `identity:get_user(user_id=ef91f0d46059481db5f9a5a0eb635aae)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:22.632073 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5cd522-04d9-4310-8a66-d81cd7228089 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:22.633863 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 120/242] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Jun 10 12:44:22 2022] GET /identity/v3/users/ef91f0d46059481db5f9a5a0eb635aae => generated 297 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:22.639084 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cf46f3-56cf-48c2-b61a-7cf1289eee6a None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:22.639084 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cf46f3-56cf-48c2-b61a-7cf1289eee6a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:22.639084 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cf46f3-56cf-48c2-b61a-7cf1289eee6a None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:22.641660 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 121/243] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:44:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:44:22.642984 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1e03be-9572-4f09-bc54-5f2debbaae90 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:22.643275 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1e03be-9572-4f09-bc54-5f2debbaae90 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:22.643554 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1e03be-9572-4f09-bc54-5f2debbaae90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:22.663061 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-fb1e03be-9572-4f09-bc54-5f2debbaae90 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:44:22.664266 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-fb1e03be-9572-4f09-bc54-5f2debbaae90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:44:22.688777 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fa22d6-ee10-4cef-b745-050941d9c633 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:22.736627 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 122/244] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:44:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 10 12:44:22.737656 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1e03be-9572-4f09-bc54-5f2debbaae90 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:22.737991 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356e52aa-3aaf-4a46-9944-efe44fde42d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:22.738883 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356e52aa-3aaf-4a46-9944-efe44fde42d6 None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:22.739335 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356e52aa-3aaf-4a46-9944-efe44fde42d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:22.739777 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356e52aa-3aaf-4a46-9944-efe44fde42d6 None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:22.741904 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356e52aa-3aaf-4a46-9944-efe44fde42d6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:22.743527 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356e52aa-3aaf-4a46-9944-efe44fde42d6 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:22.747127 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:22.839113 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 123/245] 10.222.0.27 () {60 vars in 1132 bytes} [Fri Jun 10 12:44:22 2022] POST /identity/v3/users => generated 293 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 12:44:22.840575 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356e52aa-3aaf-4a46-9944-efe44fde42d6 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:22.845256 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:22.878231 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1e03be-9572-4f09-bc54-5f2debbaae90 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:22.958557 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356e52aa-3aaf-4a46-9944-efe44fde42d6 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:22.996796 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a205ae2f-19b3-464f-b522-fc6680ccc309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:22.997981 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a205ae2f-19b3-464f-b522-fc6680ccc309 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:22.998468 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a205ae2f-19b3-464f-b522-fc6680ccc309 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:22.998945 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a205ae2f-19b3-464f-b522-fc6680ccc309 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:23.011465 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a205ae2f-19b3-464f-b522-fc6680ccc309 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:44:23.014122 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663d63ed-d321-4fc5-a186-b9ceb7b81ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:23.014791 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663d63ed-d321-4fc5-a186-b9ceb7b81ef2 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:23.015078 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663d63ed-d321-4fc5-a186-b9ceb7b81ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:23.015356 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663d63ed-d321-4fc5-a186-b9ceb7b81ef2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:23.020995 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-a205ae2f-19b3-464f-b522-fc6680ccc309 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 10 12:44:23.020995 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:44:23.020995 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:44:23.020995 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:44:23.020995 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:44:23.020995 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:44:23.020995 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:44:23.020995 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:44:23.020995 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:44:23.020995 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:44:23.020995 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:44:23.020995 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:44:23.020995 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:44:23.020995 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:44:23.020995 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:44:23.020995 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:44:23.021900 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:44:23.021900 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:44:23.021900 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:44:23.021900 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:44:23.021900 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:44:23.021900 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:44:23.021900 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:44:23.021900 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:44:23.021900 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:44:23.021900 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:44:23.021900 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:44:23.021900 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:44:23.021900 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:44:23.021900 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:44:23.021900 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:44:23.021900 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:44:23.022712 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:44:23.022712 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:44:23.022712 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:44:23.022712 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:44:23.022712 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:44:23.022712 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:44:23.022712 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:44:23.022712 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:44:23.022712 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:44:23.022712 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:44:23.022712 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 10 12:44:23.022712 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:44:23.025247 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 124/246] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Jun 10 12:44:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:44:23.036650 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a205ae2f-19b3-464f-b522-fc6680ccc309 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:23.098057 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663d63ed-d321-4fc5-a186-b9ceb7b81ef2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:23.099911 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663d63ed-d321-4fc5-a186-b9ceb7b81ef2 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:23.108881 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:23.123893 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:23.165602 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a205ae2f-19b3-464f-b522-fc6680ccc309 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:23.203664 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1e3a32-56bb-41e2-9502-50cdd6f6e039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:23.204373 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1e3a32-56bb-41e2-9502-50cdd6f6e039 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:23.204618 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1e3a32-56bb-41e2-9502-50cdd6f6e039 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:23.204908 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1e3a32-56bb-41e2-9502-50cdd6f6e039 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:23.205660 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1e3a32-56bb-41e2-9502-50cdd6f6e039 None admin] RBAC: Adding query filter params (name=member) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:23.205970 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1e3a32-56bb-41e2-9502-50cdd6f6e039 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:23.207617 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1e3a32-56bb-41e2-9502-50cdd6f6e039 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:23.211644 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 123/247] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Jun 10 12:44:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 10 12:44:23.218186 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 125/248] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Jun 10 12:44:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:23.221523 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663d63ed-d321-4fc5-a186-b9ceb7b81ef2 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:23.223408 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1e3a32-56bb-41e2-9502-50cdd6f6e039 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:23.327800 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663d63ed-d321-4fc5-a186-b9ceb7b81ef2 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:23.333388 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1e3a32-56bb-41e2-9502-50cdd6f6e039 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:23.336198 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:23.360704 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:23.365762 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98856581-8f66-441b-80c9-7f9eea580a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:23.366868 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98856581-8f66-441b-80c9-7f9eea580a65 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:23.367252 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98856581-8f66-441b-80c9-7f9eea580a65 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:23.367634 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98856581-8f66-441b-80c9-7f9eea580a65 None admin] PATH_INFO: `/v3/projects/485e375bd3394086be6c5cd49bd19e2d` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:23.369249 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98856581-8f66-441b-80c9-7f9eea580a65 None admin] RBAC: Authorizing `identity:get_project(project_id=485e375bd3394086be6c5cd49bd19e2d)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:23.370128 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9841e280-0d64-4e35-be16-29662f53f55e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:23.370807 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98856581-8f66-441b-80c9-7f9eea580a65 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:23.371132 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9841e280-0d64-4e35-be16-29662f53f55e None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:23.371490 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9841e280-0d64-4e35-be16-29662f53f55e None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:23.371842 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9841e280-0d64-4e35-be16-29662f53f55e None admin] PATH_INFO: `/v3/users/62cb9763157a443ab6b016f817628b2e` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:23.373879 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9841e280-0d64-4e35-be16-29662f53f55e None admin] RBAC: Authorizing `identity:get_user(user_id=62cb9763157a443ab6b016f817628b2e)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:23.375407 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9841e280-0d64-4e35-be16-29662f53f55e None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:23.385602 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 126/249] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Jun 10 12:44:23 2022] GET /identity/v3/users/62cb9763157a443ab6b016f817628b2e => generated 297 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 12:44:23.385725 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 124/250] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Jun 10 12:44:23 2022] GET /identity/v3/projects/485e375bd3394086be6c5cd49bd19e2d => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 12:44:23.385802 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9841e280-0d64-4e35-be16-29662f53f55e None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:23.385871 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98856581-8f66-441b-80c9-7f9eea580a65 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:23.493474 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98856581-8f66-441b-80c9-7f9eea580a65 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:23.493692 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9841e280-0d64-4e35-be16-29662f53f55e None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:23.530869 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38078bc9-28b1-4ec5-839f-78a6519e537f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:23.531540 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38078bc9-28b1-4ec5-839f-78a6519e537f None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:23.531825 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38078bc9-28b1-4ec5-839f-78a6519e537f None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:23.532135 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38078bc9-28b1-4ec5-839f-78a6519e537f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:23.532460 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3de1f37-7c78-46be-a0fe-050d1d062ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:23.532989 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38078bc9-28b1-4ec5-839f-78a6519e537f None admin] RBAC: Adding query filter params (role.id=7d6c4b2adb9d43bca1a6573e8f7a9381, scope.project.id=485e375bd3394086be6c5cd49bd19e2d, user.id=ef91f0d46059481db5f9a5a0eb635aae) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:23.533305 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38078bc9-28b1-4ec5-839f-78a6519e537f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:23.533400 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3de1f37-7c78-46be-a0fe-050d1d062ac1 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:23.533595 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3de1f37-7c78-46be-a0fe-050d1d062ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:23.533922 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3de1f37-7c78-46be-a0fe-050d1d062ac1 None admin] PATH_INFO: `/v3/projects/485e375bd3394086be6c5cd49bd19e2d` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:23.534763 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38078bc9-28b1-4ec5-839f-78a6519e537f None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:23.535106 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3de1f37-7c78-46be-a0fe-050d1d062ac1 None admin] RBAC: Authorizing `identity:get_project(project_id=485e375bd3394086be6c5cd49bd19e2d)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:23.536565 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3de1f37-7c78-46be-a0fe-050d1d062ac1 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:23.538901 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 125/251] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Jun 10 12:44:23 2022] GET /identity/v3/projects/485e375bd3394086be6c5cd49bd19e2d => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:23.544733 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3de1f37-7c78-46be-a0fe-050d1d062ac1 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:23.545795 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 127/252] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Jun 10 12:44:23 2022] GET /identity/v3/role_assignments?user.id=ef91f0d46059481db5f9a5a0eb635aae&scope.project.id=485e375bd3394086be6c5cd49bd19e2d&role.id=7d6c4b2adb9d43bca1a6573e8f7a9381 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 12:44:23.594516 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:23.659928 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3de1f37-7c78-46be-a0fe-050d1d062ac1 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:23.699234 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc821e0-34f4-484e-9bfc-82f6d92ba90e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:23.700279 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc821e0-34f4-484e-9bfc-82f6d92ba90e None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:23.700637 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc821e0-34f4-484e-9bfc-82f6d92ba90e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:23.701047 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc821e0-34f4-484e-9bfc-82f6d92ba90e None admin] PATH_INFO: `/v3/projects/485e375bd3394086be6c5cd49bd19e2d/users/62cb9763157a443ab6b016f817628b2e/roles/43106c664e1b40de984c0c62701149c8` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:23.703211 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc821e0-34f4-484e-9bfc-82f6d92ba90e None admin] RBAC: Authorizing `identity:create_grant(project_id=485e375bd3394086be6c5cd49bd19e2d, user_id=62cb9763157a443ab6b016f817628b2e, role_id=43106c664e1b40de984c0c62701149c8)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:23.704771 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc821e0-34f4-484e-9bfc-82f6d92ba90e None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:23.775332 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 126/253] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Jun 10 12:44:23 2022] PUT /identity/v3/projects/485e375bd3394086be6c5cd49bd19e2d/users/62cb9763157a443ab6b016f817628b2e/roles/43106c664e1b40de984c0c62701149c8 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 12:44:23.836595 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86b8c62-d385-49b1-87dc-fbbbc2b9286e None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:23.836897 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86b8c62-d385-49b1-87dc-fbbbc2b9286e None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:23.837171 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86b8c62-d385-49b1-87dc-fbbbc2b9286e None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:23.838656 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 128/254] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:44:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:44:23.844640 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db944fe7-2579-4e9c-9d89-3481515ad711 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:23.844935 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db944fe7-2579-4e9c-9d89-3481515ad711 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:23.845206 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db944fe7-2579-4e9c-9d89-3481515ad711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:23.870187 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-db944fe7-2579-4e9c-9d89-3481515ad711 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:44:23.871455 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-db944fe7-2579-4e9c-9d89-3481515ad711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:44:23.935944 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e986dca3-f8e2-4b2c-8221-9da0e5007125 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:23.935944 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e986dca3-f8e2-4b2c-8221-9da0e5007125 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:23.935944 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e986dca3-f8e2-4b2c-8221-9da0e5007125 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:23.937511 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 129/255] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:44:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 10 12:44:23.942971 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd700ecc-0031-4091-bf34-fe1ccdf00fda None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:23.943272 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd700ecc-0031-4091-bf34-fe1ccdf00fda None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:23.943534 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd700ecc-0031-4091-bf34-fe1ccdf00fda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:23.962082 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-cd700ecc-0031-4091-bf34-fe1ccdf00fda None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:44:23.963454 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-cd700ecc-0031-4091-bf34-fe1ccdf00fda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:44:23.995312 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 127/256] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:44:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 10 12:44:24.033326 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db944fe7-2579-4e9c-9d89-3481515ad711 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:24.052329 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:24.078042 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 130/257] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:44:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 10 12:44:24.108248 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:24.140439 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd700ecc-0031-4091-bf34-fe1ccdf00fda None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:24.190166 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db944fe7-2579-4e9c-9d89-3481515ad711 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:24.264749 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd700ecc-0031-4091-bf34-fe1ccdf00fda None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:24.283279 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee3e05d-fc18-4e43-a934-adc70f4b7ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:24.284191 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee3e05d-fc18-4e43-a934-adc70f4b7ac3 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:24.284475 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee3e05d-fc18-4e43-a934-adc70f4b7ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:24.284750 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee3e05d-fc18-4e43-a934-adc70f4b7ac3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:24.300519 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dd87b5-cd1f-4fe6-ac9a-cae51b8f6c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:24.301327 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dd87b5-cd1f-4fe6-ac9a-cae51b8f6c46 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:24.301611 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dd87b5-cd1f-4fe6-ac9a-cae51b8f6c46 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:24.301892 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dd87b5-cd1f-4fe6-ac9a-cae51b8f6c46 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:24.302015 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee3e05d-fc18-4e43-a934-adc70f4b7ac3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:44:24.313083 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dd87b5-cd1f-4fe6-ac9a-cae51b8f6c46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:44:24.322527 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-2ee3e05d-fc18-4e43-a934-adc70f4b7ac3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 10 12:44:24.322527 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:44:24.322527 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:44:24.322527 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:44:24.322527 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:44:24.322527 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:44:24.322527 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:44:24.322527 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:44:24.322527 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:44:24.322527 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:44:24.322527 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:44:24.322527 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:44:24.322527 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:44:24.322527 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:44:24.322527 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:44:24.322527 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:44:24.323454 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:44:24.323454 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:44:24.323454 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:44:24.323454 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:44:24.323454 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:44:24.323454 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:44:24.323454 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:44:24.323454 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:44:24.323454 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:44:24.323454 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:44:24.323454 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:44:24.323454 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:44:24.323454 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:44:24.323454 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:44:24.323454 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:44:24.323454 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:44:24.324305 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:44:24.324305 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:44:24.324305 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:44:24.324305 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:44:24.324305 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:44:24.324305 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:44:24.324305 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:44:24.324305 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:44:24.324305 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:44:24.324305 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:44:24.324305 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 10 12:44:24.324305 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:44:24.324305 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 128/258] 10.222.0.27 () {56 vars in 1114 bytes} [Fri Jun 10 12:44:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:44:24.326509 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-48dd87b5-cd1f-4fe6-ac9a-cae51b8f6c46 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 10 12:44:24.326509 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:44:24.326509 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:44:24.326509 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:44:24.326509 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:44:24.326509 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:44:24.326509 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:44:24.326509 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:44:24.326509 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:44:24.326509 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:44:24.326509 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:44:24.326509 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:44:24.326509 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:44:24.326509 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:44:24.326509 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:44:24.326509 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:44:24.327373 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:44:24.327373 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:44:24.327373 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:44:24.327373 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:44:24.327373 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:44:24.327373 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:44:24.327373 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:44:24.327373 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:44:24.327373 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:44:24.327373 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:44:24.327373 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:44:24.327373 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:44:24.327373 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:44:24.327373 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:44:24.327373 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:44:24.327373 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:44:24.328311 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:44:24.328311 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:44:24.328311 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:44:24.328311 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:44:24.328311 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:44:24.328311 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:44:24.328311 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:44:24.328311 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:44:24.328311 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:44:24.328311 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:44:24.328311 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 10 12:44:24.328311 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:44:24.331181 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee3e05d-fc18-4e43-a934-adc70f4b7ac3 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:24.331380 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 131/259] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Jun 10 12:44:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:44:24.337259 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dd87b5-cd1f-4fe6-ac9a-cae51b8f6c46 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:24.408824 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:24.484334 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:24.487453 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dd87b5-cd1f-4fe6-ac9a-cae51b8f6c46 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:24.498980 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee3e05d-fc18-4e43-a934-adc70f4b7ac3 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:24.542884 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753bd1d3-dfcb-497e-9b49-4cb78920c3bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:24.543572 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753bd1d3-dfcb-497e-9b49-4cb78920c3bd None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:24.543852 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753bd1d3-dfcb-497e-9b49-4cb78920c3bd None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:24.544150 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753bd1d3-dfcb-497e-9b49-4cb78920c3bd None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:24.544909 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753bd1d3-dfcb-497e-9b49-4cb78920c3bd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:24.545213 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753bd1d3-dfcb-497e-9b49-4cb78920c3bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:24.546610 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753bd1d3-dfcb-497e-9b49-4cb78920c3bd None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:24.572325 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 129/260] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Jun 10 12:44:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 12:44:24.573155 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c9cc92-ea94-40d2-8fde-5ade89de88fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:24.573896 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c9cc92-ea94-40d2-8fde-5ade89de88fd None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:24.574180 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c9cc92-ea94-40d2-8fde-5ade89de88fd None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:24.574454 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c9cc92-ea94-40d2-8fde-5ade89de88fd None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:24.575209 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c9cc92-ea94-40d2-8fde-5ade89de88fd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:24.575513 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c9cc92-ea94-40d2-8fde-5ade89de88fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:24.577765 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753bd1d3-dfcb-497e-9b49-4cb78920c3bd None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:24.578444 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c9cc92-ea94-40d2-8fde-5ade89de88fd None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:24.598568 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 132/261] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Jun 10 12:44:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 12:44:24.604400 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c9cc92-ea94-40d2-8fde-5ade89de88fd None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:24.814521 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c9cc92-ea94-40d2-8fde-5ade89de88fd None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:24.834292 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:24.863695 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753bd1d3-dfcb-497e-9b49-4cb78920c3bd None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:24.936557 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65a67ae-0267-4990-a605-7b2df489c39f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:24.937437 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65a67ae-0267-4990-a605-7b2df489c39f None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:24.937719 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65a67ae-0267-4990-a605-7b2df489c39f None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:24.937993 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65a67ae-0267-4990-a605-7b2df489c39f None admin] PATH_INFO: `/v3/users/7576e331fed7406ab308506ab4e67edb` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:24.939221 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65a67ae-0267-4990-a605-7b2df489c39f None admin] RBAC: Authorizing `identity:get_user(user_id=7576e331fed7406ab308506ab4e67edb)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:24.940690 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65a67ae-0267-4990-a605-7b2df489c39f None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:24.942307 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 133/262] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Jun 10 12:44:24 2022] GET /identity/v3/users/7576e331fed7406ab308506ab4e67edb => generated 283 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:24.948854 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65a67ae-0267-4990-a605-7b2df489c39f None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:24.955247 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4e0ec7-e169-4934-8c0b-a92c790ec4ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:24.956214 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4e0ec7-e169-4934-8c0b-a92c790ec4ff None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:24.956505 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4e0ec7-e169-4934-8c0b-a92c790ec4ff None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:24.956784 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4e0ec7-e169-4934-8c0b-a92c790ec4ff None admin] PATH_INFO: `/v3/users/7576e331fed7406ab308506ab4e67edb` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:24.960579 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4e0ec7-e169-4934-8c0b-a92c790ec4ff None admin] RBAC: Authorizing `identity:get_user(user_id=7576e331fed7406ab308506ab4e67edb)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:24.962040 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4e0ec7-e169-4934-8c0b-a92c790ec4ff None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:24.968078 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 130/263] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Jun 10 12:44:24 2022] GET /identity/v3/users/7576e331fed7406ab308506ab4e67edb => generated 283 bytes in 386 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:24.973029 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4e0ec7-e169-4934-8c0b-a92c790ec4ff None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:25.028208 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:25.148779 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4e0ec7-e169-4934-8c0b-a92c790ec4ff None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:25.209594 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2955faf3-132f-4428-a9e8-1ad6c80c3d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:25.210664 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2955faf3-132f-4428-a9e8-1ad6c80c3d46 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:25.211153 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2955faf3-132f-4428-a9e8-1ad6c80c3d46 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:25.211656 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2955faf3-132f-4428-a9e8-1ad6c80c3d46 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:25.262506 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65a67ae-0267-4990-a605-7b2df489c39f None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:25.356565 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae64af3-7672-4347-a70c-a954725de10d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:25.357226 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae64af3-7672-4347-a70c-a954725de10d None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:25.357512 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae64af3-7672-4347-a70c-a954725de10d None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:25.357792 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae64af3-7672-4347-a70c-a954725de10d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:25.364927 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2955faf3-132f-4428-a9e8-1ad6c80c3d46 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:25.366415 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2955faf3-132f-4428-a9e8-1ad6c80c3d46 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:25.420306 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:25.484198 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:25.540673 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae64af3-7672-4347-a70c-a954725de10d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:25.542677 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae64af3-7672-4347-a70c-a954725de10d None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:25.609843 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 131/264] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Jun 10 12:44:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 637 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 10 12:44:25.615342 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2955faf3-132f-4428-a9e8-1ad6c80c3d46 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:25.767458 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 134/265] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Jun 10 12:44:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 819 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 10 12:44:25.773962 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae64af3-7672-4347-a70c-a954725de10d None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:25.816851 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2955faf3-132f-4428-a9e8-1ad6c80c3d46 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:25.857472 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:25.879307 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97e8fa1-9f14-4bf0-875d-3151f8e6e2e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:25.880207 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97e8fa1-9f14-4bf0-875d-3151f8e6e2e7 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:25.880495 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97e8fa1-9f14-4bf0-875d-3151f8e6e2e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:25.880778 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97e8fa1-9f14-4bf0-875d-3151f8e6e2e7 None admin] PATH_INFO: `/v3/projects/485e375bd3394086be6c5cd49bd19e2d` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:25.882564 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97e8fa1-9f14-4bf0-875d-3151f8e6e2e7 None admin] RBAC: Authorizing `identity:get_project(project_id=485e375bd3394086be6c5cd49bd19e2d)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:25.884040 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97e8fa1-9f14-4bf0-875d-3151f8e6e2e7 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:25.889218 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 132/266] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Jun 10 12:44:25 2022] GET /identity/v3/projects/485e375bd3394086be6c5cd49bd19e2d => generated 302 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:25.892073 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97e8fa1-9f14-4bf0-875d-3151f8e6e2e7 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:25.911957 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:25.950666 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae64af3-7672-4347-a70c-a954725de10d None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:26.016572 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79645677-7d42-440c-b2cf-8240bc7b455c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:26.017560 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79645677-7d42-440c-b2cf-8240bc7b455c None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:26.017840 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79645677-7d42-440c-b2cf-8240bc7b455c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:26.018125 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79645677-7d42-440c-b2cf-8240bc7b455c None admin] PATH_INFO: `/v3/projects/485e375bd3394086be6c5cd49bd19e2d` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:26.020581 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79645677-7d42-440c-b2cf-8240bc7b455c None admin] RBAC: Authorizing `identity:get_project(project_id=485e375bd3394086be6c5cd49bd19e2d)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:26.024098 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79645677-7d42-440c-b2cf-8240bc7b455c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:26.026043 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 135/267] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Jun 10 12:44:25 2022] GET /identity/v3/projects/485e375bd3394086be6c5cd49bd19e2d => generated 302 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 12:44:26.034908 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79645677-7d42-440c-b2cf-8240bc7b455c None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:26.074276 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97e8fa1-9f14-4bf0-875d-3151f8e6e2e7 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:26.129191 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d974523-5160-401f-802f-a05bfd86229e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:26.129871 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d974523-5160-401f-802f-a05bfd86229e None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:26.130160 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d974523-5160-401f-802f-a05bfd86229e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:26.130442 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d974523-5160-401f-802f-a05bfd86229e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:26.131295 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d974523-5160-401f-802f-a05bfd86229e None admin] RBAC: Adding query filter params (role.id=2d71ee7c2a544fe5ad77d92b24280efe, scope.project.id=485e375bd3394086be6c5cd49bd19e2d, user.id=7576e331fed7406ab308506ab4e67edb) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:26.131609 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d974523-5160-401f-802f-a05bfd86229e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:26.133119 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d974523-5160-401f-802f-a05bfd86229e None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:26.153959 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 133/268] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Jun 10 12:44:25 2022] GET /identity/v3/role_assignments?user.id=7576e331fed7406ab308506ab4e67edb&scope.project.id=485e375bd3394086be6c5cd49bd19e2d&role.id=2d71ee7c2a544fe5ad77d92b24280efe => generated 608 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:26.221755 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79645677-7d42-440c-b2cf-8240bc7b455c None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:26.260215 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ade29ec-40db-4cc4-bfd1-f0a253915910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:26.260890 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ade29ec-40db-4cc4-bfd1-f0a253915910 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:26.261178 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ade29ec-40db-4cc4-bfd1-f0a253915910 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:26.261456 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ade29ec-40db-4cc4-bfd1-f0a253915910 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:26.262303 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ade29ec-40db-4cc4-bfd1-f0a253915910 None admin] RBAC: Adding query filter params (role.id=b43d5a8247e244869c2c70f986f42e8a, scope.project.id=485e375bd3394086be6c5cd49bd19e2d, user.id=7576e331fed7406ab308506ab4e67edb) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:26.262615 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ade29ec-40db-4cc4-bfd1-f0a253915910 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:26.264090 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ade29ec-40db-4cc4-bfd1-f0a253915910 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:26.271139 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 136/269] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Jun 10 12:44:26 2022] GET /identity/v3/role_assignments?user.id=7576e331fed7406ab308506ab4e67edb&scope.project.id=485e375bd3394086be6c5cd49bd19e2d&role.id=b43d5a8247e244869c2c70f986f42e8a => generated 608 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:26.537488 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4679b878-7b89-4b51-ade0-e4459685ed41 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:26.537838 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4679b878-7b89-4b51-ade0-e4459685ed41 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:26.538157 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4679b878-7b89-4b51-ade0-e4459685ed41 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:26.539534 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 134/270] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:44:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:44:26.545484 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21af50f7-8d50-4a11-9c1f-9fd4d5029ef7 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:26.545778 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21af50f7-8d50-4a11-9c1f-9fd4d5029ef7 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:26.546064 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21af50f7-8d50-4a11-9c1f-9fd4d5029ef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:26.564581 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-21af50f7-8d50-4a11-9c1f-9fd4d5029ef7 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:44:26.565789 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-21af50f7-8d50-4a11-9c1f-9fd4d5029ef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:44:26.568848 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:26.644401 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 137/271] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:44:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 10 12:44:26.671540 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4679b878-7b89-4b51-ade0-e4459685ed41 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:26.683112 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff2e39e-ab9b-4d93-94f4-c57397ea69bf None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:26.683418 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff2e39e-ab9b-4d93-94f4-c57397ea69bf None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:26.683683 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff2e39e-ab9b-4d93-94f4-c57397ea69bf None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:26.685599 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 138/272] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:44:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:44:26.693638 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0cc376-83d8-4422-914e-896dfdbdf982 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:26.693924 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0cc376-83d8-4422-914e-896dfdbdf982 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:26.694279 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0cc376-83d8-4422-914e-896dfdbdf982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:26.715543 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-ed0cc376-83d8-4422-914e-896dfdbdf982 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:44:26.716767 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-ed0cc376-83d8-4422-914e-896dfdbdf982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:44:26.749799 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:26.807787 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4679b878-7b89-4b51-ade0-e4459685ed41 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:26.809419 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 139/273] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:44:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 10 12:44:26.810956 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ae1156-247c-483e-b01e-8ca9c2cb39da None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:26.811522 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ae1156-247c-483e-b01e-8ca9c2cb39da None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:26.812064 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ae1156-247c-483e-b01e-8ca9c2cb39da None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:26.813576 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 140/274] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:44:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:44:26.819595 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e142e6-d571-42f8-af30-520b603d73c3 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:26.820096 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e142e6-d571-42f8-af30-520b603d73c3 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:26.820520 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e142e6-d571-42f8-af30-520b603d73c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:26.839731 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-c6e142e6-d571-42f8-af30-520b603d73c3 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:44:26.840949 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c6e142e6-d571-42f8-af30-520b603d73c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:44:26.854021 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c9cb8a-5411-4ed4-975d-e1db217195f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:26.854820 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c9cb8a-5411-4ed4-975d-e1db217195f7 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:26.855101 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c9cb8a-5411-4ed4-975d-e1db217195f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:26.855376 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c9cb8a-5411-4ed4-975d-e1db217195f7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:26.856555 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c9cb8a-5411-4ed4-975d-e1db217195f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:26.858010 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c9cb8a-5411-4ed4-975d-e1db217195f7 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:26.859615 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 135/275] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:44:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:26.860937 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efb7417-d5af-4325-b15f-63bc8f1781dc None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:26.861223 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efb7417-d5af-4325-b15f-63bc8f1781dc None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:26.861502 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efb7417-d5af-4325-b15f-63bc8f1781dc None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:26.865199 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 136/276] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:44:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:44:26.865199 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efb7417-d5af-4325-b15f-63bc8f1781dc None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:26.913123 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:26.918232 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 141/277] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:44:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 10 12:44:26.919013 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e142e6-d571-42f8-af30-520b603d73c3 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:26.966950 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efb7417-d5af-4325-b15f-63bc8f1781dc None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:26.989414 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:27.001059 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310e7122-2376-45dd-a553-2ded101bce1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:27.001843 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310e7122-2376-45dd-a553-2ded101bce1b None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:27.003962 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310e7122-2376-45dd-a553-2ded101bce1b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:27.003962 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310e7122-2376-45dd-a553-2ded101bce1b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:27.013350 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310e7122-2376-45dd-a553-2ded101bce1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:44:27.016755 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e142e6-d571-42f8-af30-520b603d73c3 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:27.021463 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-310e7122-2376-45dd-a553-2ded101bce1b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 10 12:44:27.021463 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:44:27.021463 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:44:27.021463 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:44:27.021463 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:44:27.021463 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:44:27.021463 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:44:27.021463 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:44:27.021463 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:44:27.021463 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:44:27.021463 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:44:27.021463 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:44:27.021463 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:44:27.021463 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:44:27.021463 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:44:27.021463 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:44:27.022544 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:44:27.022544 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:44:27.022544 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:44:27.022544 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:44:27.022544 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:44:27.022544 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:44:27.022544 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:44:27.022544 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:44:27.022544 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:44:27.022544 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:44:27.022544 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:44:27.022544 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:44:27.022544 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:44:27.022544 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:44:27.022544 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:44:27.022544 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:44:27.023721 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:44:27.023721 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:44:27.023721 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:44:27.023721 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:44:27.023721 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:44:27.023721 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:44:27.023721 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:44:27.023721 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:44:27.023721 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:44:27.023721 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:44:27.023721 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 10 12:44:27.023721 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:44:27.023721 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 137/278] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Jun 10 12:44:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:44:27.024716 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6c85b5-c089-4830-be34-49f7067347a2 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:27.024716 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6c85b5-c089-4830-be34-49f7067347a2 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:27.024716 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6c85b5-c089-4830-be34-49f7067347a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:27.040416 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-2a6c85b5-c089-4830-be34-49f7067347a2 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:44:27.042135 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2a6c85b5-c089-4830-be34-49f7067347a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:44:27.051367 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479b7884-454c-4cd4-8756-5825038e8383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:27.052062 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479b7884-454c-4cd4-8756-5825038e8383 None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:27.052349 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479b7884-454c-4cd4-8756-5825038e8383 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:27.052630 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479b7884-454c-4cd4-8756-5825038e8383 None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:27.053326 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479b7884-454c-4cd4-8756-5825038e8383 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:27.054763 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479b7884-454c-4cd4-8756-5825038e8383 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:27.107022 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 138/279] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:44:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 10 12:44:27.107773 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6c85b5-c089-4830-be34-49f7067347a2 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:27.145279 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 142/280] 10.222.0.27 () {60 vars in 1132 bytes} [Fri Jun 10 12:44:26 2022] POST /identity/v3/users => generated 293 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 12:44:27.146204 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479b7884-454c-4cd4-8756-5825038e8383 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:27.213484 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:27.218032 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6c85b5-c089-4830-be34-49f7067347a2 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:27.235267 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:27.258468 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43c2e00-308f-4b35-971d-ca200ee29544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:27.259445 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43c2e00-308f-4b35-971d-ca200ee29544 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:27.259832 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43c2e00-308f-4b35-971d-ca200ee29544 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:27.260222 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43c2e00-308f-4b35-971d-ca200ee29544 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:27.261896 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479b7884-454c-4cd4-8756-5825038e8383 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:27.271787 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43c2e00-308f-4b35-971d-ca200ee29544 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:44:27.279973 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-b43c2e00-308f-4b35-971d-ca200ee29544 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 10 12:44:27.279973 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:44:27.279973 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:44:27.279973 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:44:27.279973 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:44:27.279973 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:44:27.279973 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:44:27.279973 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:44:27.279973 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:44:27.279973 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:44:27.279973 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:44:27.279973 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:44:27.279973 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:44:27.279973 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:44:27.279973 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:44:27.279973 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:44:27.280957 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:44:27.280957 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:44:27.280957 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:44:27.280957 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:44:27.280957 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:44:27.280957 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:44:27.280957 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:44:27.280957 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:44:27.280957 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:44:27.280957 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:44:27.280957 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:44:27.280957 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:44:27.280957 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:44:27.280957 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:44:27.280957 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:44:27.280957 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:44:27.281818 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:44:27.281818 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:44:27.281818 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:44:27.281818 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:44:27.281818 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:44:27.281818 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:44:27.281818 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:44:27.281818 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:44:27.281818 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:44:27.281818 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:44:27.281818 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 10 12:44:27.281818 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:44:27.281818 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 139/281] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Jun 10 12:44:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:44:27.282476 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43c2e00-308f-4b35-971d-ca200ee29544 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:27.299735 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2e0aa6-3b35-4174-9c39-fdcaab6e2b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:27.300477 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2e0aa6-3b35-4174-9c39-fdcaab6e2b9d None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:27.301393 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2e0aa6-3b35-4174-9c39-fdcaab6e2b9d None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:27.301393 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2e0aa6-3b35-4174-9c39-fdcaab6e2b9d None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:27.301798 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2e0aa6-3b35-4174-9c39-fdcaab6e2b9d None admin] RBAC: Adding query filter params (name=member) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:27.302107 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2e0aa6-3b35-4174-9c39-fdcaab6e2b9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:27.303521 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2e0aa6-3b35-4174-9c39-fdcaab6e2b9d None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:27.316047 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 143/282] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Jun 10 12:44:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:27.316047 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2e0aa6-3b35-4174-9c39-fdcaab6e2b9d None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:27.404261 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43c2e00-308f-4b35-971d-ca200ee29544 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:27.442353 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2e0aa6-3b35-4174-9c39-fdcaab6e2b9d None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:27.445362 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1a97b3-0f5d-41e2-ad70-f79107fedc40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:27.446280 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1a97b3-0f5d-41e2-ad70-f79107fedc40 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:27.446702 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1a97b3-0f5d-41e2-ad70-f79107fedc40 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:27.447044 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1a97b3-0f5d-41e2-ad70-f79107fedc40 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:27.460807 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1a97b3-0f5d-41e2-ad70-f79107fedc40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:44:27.468334 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:27.469430 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-5b1a97b3-0f5d-41e2-ad70-f79107fedc40 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 10 12:44:27.469430 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:44:27.469430 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:44:27.469430 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:44:27.469430 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:44:27.469430 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:44:27.469430 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:44:27.469430 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:44:27.469430 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:44:27.469430 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:44:27.469430 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:44:27.469430 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:44:27.469430 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:44:27.469430 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:44:27.469430 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:44:27.469430 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:44:27.470515 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:44:27.470515 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:44:27.470515 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:44:27.470515 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:44:27.470515 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:44:27.470515 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:44:27.470515 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:44:27.470515 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:44:27.470515 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:44:27.470515 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:44:27.470515 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:44:27.470515 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:44:27.470515 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:44:27.470515 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:44:27.470515 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:44:27.470515 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:44:27.471435 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:44:27.471435 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:44:27.471435 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:44:27.471435 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:44:27.471435 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:44:27.471435 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:44:27.471435 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:44:27.471435 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:44:27.471435 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:44:27.471435 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:44:27.471435 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 10 12:44:27.471435 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:44:27.471435 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 140/283] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Jun 10 12:44:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:44:27.472263 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1a97b3-0f5d-41e2-ad70-f79107fedc40 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:27.487656 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c380ce-f88d-45ac-81e9-c99bafdeb352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:27.488513 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c380ce-f88d-45ac-81e9-c99bafdeb352 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:27.488849 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c380ce-f88d-45ac-81e9-c99bafdeb352 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:27.489124 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c380ce-f88d-45ac-81e9-c99bafdeb352 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:27.489924 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c380ce-f88d-45ac-81e9-c99bafdeb352 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:27.490231 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c380ce-f88d-45ac-81e9-c99bafdeb352 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:27.491723 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c380ce-f88d-45ac-81e9-c99bafdeb352 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:27.502377 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 144/284] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Jun 10 12:44:27 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:27.503163 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c380ce-f88d-45ac-81e9-c99bafdeb352 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:27.513642 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:27.593590 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1a97b3-0f5d-41e2-ad70-f79107fedc40 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:27.632105 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8728efc8-197e-4c6d-b64e-e50301a4cc14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:27.633438 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8728efc8-197e-4c6d-b64e-e50301a4cc14 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:27.633956 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8728efc8-197e-4c6d-b64e-e50301a4cc14 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:27.634415 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8728efc8-197e-4c6d-b64e-e50301a4cc14 None admin] PATH_INFO: `/v3/users/1ac03230095448b2bd230560807af974` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:27.636866 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c380ce-f88d-45ac-81e9-c99bafdeb352 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:27.653583 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8728efc8-197e-4c6d-b64e-e50301a4cc14 None admin] RBAC: Authorizing `identity:get_user(user_id=1ac03230095448b2bd230560807af974)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:27.655344 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8728efc8-197e-4c6d-b64e-e50301a4cc14 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:27.661958 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 141/285] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Jun 10 12:44:27 2022] GET /identity/v3/users/1ac03230095448b2bd230560807af974 => generated 293 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:27.661958 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8728efc8-197e-4c6d-b64e-e50301a4cc14 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:27.680489 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4204dc-2788-441b-880e-9f10247d6005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:27.681485 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4204dc-2788-441b-880e-9f10247d6005 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:27.682037 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4204dc-2788-441b-880e-9f10247d6005 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:27.682619 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4204dc-2788-441b-880e-9f10247d6005 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:27.683593 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4204dc-2788-441b-880e-9f10247d6005 None admin] RBAC: Adding query filter params (name=member) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:27.684173 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4204dc-2788-441b-880e-9f10247d6005 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:27.685955 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4204dc-2788-441b-880e-9f10247d6005 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:27.695323 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 145/286] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Jun 10 12:44:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:27.696730 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4204dc-2788-441b-880e-9f10247d6005 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:27.743656 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:27.755983 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:27.769496 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8728efc8-197e-4c6d-b64e-e50301a4cc14 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:27.812346 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c12fa43-6927-45b8-87ce-ef1b4cb2d413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:27.813221 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c12fa43-6927-45b8-87ce-ef1b4cb2d413 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:27.813520 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c12fa43-6927-45b8-87ce-ef1b4cb2d413 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:27.813919 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c12fa43-6927-45b8-87ce-ef1b4cb2d413 None admin] PATH_INFO: `/v3/users/ef91f0d46059481db5f9a5a0eb635aae` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:27.815244 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c12fa43-6927-45b8-87ce-ef1b4cb2d413 None admin] RBAC: Authorizing `identity:get_user(user_id=ef91f0d46059481db5f9a5a0eb635aae)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:27.816736 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c12fa43-6927-45b8-87ce-ef1b4cb2d413 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:27.818987 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 142/287] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Jun 10 12:44:27 2022] GET /identity/v3/users/ef91f0d46059481db5f9a5a0eb635aae => generated 297 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:27.819565 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c12fa43-6927-45b8-87ce-ef1b4cb2d413 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:27.822882 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4204dc-2788-441b-880e-9f10247d6005 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:27.873218 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9eb9ac-96ae-4196-bdb7-d5d174bb7bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:27.874074 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9eb9ac-96ae-4196-bdb7-d5d174bb7bfc None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:27.874498 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9eb9ac-96ae-4196-bdb7-d5d174bb7bfc None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:27.874913 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9eb9ac-96ae-4196-bdb7-d5d174bb7bfc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:27.875931 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9eb9ac-96ae-4196-bdb7-d5d174bb7bfc None admin] RBAC: Adding query filter params (role.id=43106c664e1b40de984c0c62701149c8, user.id=1ac03230095448b2bd230560807af974, scope.system=all) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:27.876380 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9eb9ac-96ae-4196-bdb7-d5d174bb7bfc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:27.877960 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9eb9ac-96ae-4196-bdb7-d5d174bb7bfc None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:27.883550 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 146/288] 10.222.0.27 () {56 vars in 1308 bytes} [Fri Jun 10 12:44:27 2022] GET /identity/v3/role_assignments?user.id=1ac03230095448b2bd230560807af974&scope.system=all&role.id=43106c664e1b40de984c0c62701149c8 => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:27.884582 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9eb9ac-96ae-4196-bdb7-d5d174bb7bfc None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:27.984835 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c12fa43-6927-45b8-87ce-ef1b4cb2d413 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:28.024905 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64526e74-c506-438a-809b-d527ec27ae5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:28.026796 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64526e74-c506-438a-809b-d527ec27ae5e None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:28.027258 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64526e74-c506-438a-809b-d527ec27ae5e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:28.027773 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64526e74-c506-438a-809b-d527ec27ae5e None admin] PATH_INFO: `/v3/users/62cb9763157a443ab6b016f817628b2e` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:28.029315 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64526e74-c506-438a-809b-d527ec27ae5e None admin] RBAC: Authorizing `identity:get_user(user_id=62cb9763157a443ab6b016f817628b2e)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:28.031037 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64526e74-c506-438a-809b-d527ec27ae5e None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:28.032986 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 143/289] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Jun 10 12:44:27 2022] GET /identity/v3/users/62cb9763157a443ab6b016f817628b2e => generated 297 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:28.038817 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64526e74-c506-438a-809b-d527ec27ae5e None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:28.093535 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:28.138344 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9eb9ac-96ae-4196-bdb7-d5d174bb7bfc None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:28.155062 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:28.218916 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce86d66-7aac-4e97-98b4-d7951ca25609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:28.219788 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce86d66-7aac-4e97-98b4-d7951ca25609 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:28.220135 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce86d66-7aac-4e97-98b4-d7951ca25609 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:28.220418 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce86d66-7aac-4e97-98b4-d7951ca25609 None admin] PATH_INFO: `/v3/projects/485e375bd3394086be6c5cd49bd19e2d` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:28.221806 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce86d66-7aac-4e97-98b4-d7951ca25609 None admin] RBAC: Authorizing `identity:get_project(project_id=485e375bd3394086be6c5cd49bd19e2d)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:28.223246 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce86d66-7aac-4e97-98b4-d7951ca25609 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:28.225188 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 147/290] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Jun 10 12:44:27 2022] GET /identity/v3/projects/485e375bd3394086be6c5cd49bd19e2d => generated 302 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:28.231168 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce86d66-7aac-4e97-98b4-d7951ca25609 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:28.255945 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64526e74-c506-438a-809b-d527ec27ae5e None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:28.317521 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041d5789-486a-44b0-9aa8-234f99de15ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:28.318179 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041d5789-486a-44b0-9aa8-234f99de15ff None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:28.318465 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041d5789-486a-44b0-9aa8-234f99de15ff None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:28.318743 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041d5789-486a-44b0-9aa8-234f99de15ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:28.433616 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041d5789-486a-44b0-9aa8-234f99de15ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:28.435408 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041d5789-486a-44b0-9aa8-234f99de15ff None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:28.445689 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce86d66-7aac-4e97-98b4-d7951ca25609 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:28.478215 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:28.533873 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66387f99-4b2d-4692-90c3-81d0b4239229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:28.534687 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66387f99-4b2d-4692-90c3-81d0b4239229 None admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:28.534972 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66387f99-4b2d-4692-90c3-81d0b4239229 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:28.535251 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66387f99-4b2d-4692-90c3-81d0b4239229 None admin] PATH_INFO: `/v3/projects/485e375bd3394086be6c5cd49bd19e2d/users/ef91f0d46059481db5f9a5a0eb635aae/roles/7d6c4b2adb9d43bca1a6573e8f7a9381` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:28.538556 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66387f99-4b2d-4692-90c3-81d0b4239229 None admin] RBAC: Authorizing `identity:create_grant(project_id=485e375bd3394086be6c5cd49bd19e2d, user_id=ef91f0d46059481db5f9a5a0eb635aae, role_id=7d6c4b2adb9d43bca1a6573e8f7a9381)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:28.540048 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66387f99-4b2d-4692-90c3-81d0b4239229 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:28.591540 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 144/291] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Jun 10 12:44:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 553 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 10 12:44:28.596995 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041d5789-486a-44b0-9aa8-234f99de15ff None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:28.624294 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 148/292] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Jun 10 12:44:28 2022] PUT /identity/v3/projects/485e375bd3394086be6c5cd49bd19e2d/users/ef91f0d46059481db5f9a5a0eb635aae/roles/7d6c4b2adb9d43bca1a6573e8f7a9381 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 12:44:28.706765 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041d5789-486a-44b0-9aa8-234f99de15ff None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:28.735260 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:28.744172 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7ad725-e44d-4e9c-a886-69541a3568e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:28.745068 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7ad725-e44d-4e9c-a886-69541a3568e4 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:28.745354 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7ad725-e44d-4e9c-a886-69541a3568e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:28.745633 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7ad725-e44d-4e9c-a886-69541a3568e4 None admin] PATH_INFO: `/v3/projects/485e375bd3394086be6c5cd49bd19e2d` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:28.746932 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7ad725-e44d-4e9c-a886-69541a3568e4 None admin] RBAC: Authorizing `identity:get_project(project_id=485e375bd3394086be6c5cd49bd19e2d)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:28.748533 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7ad725-e44d-4e9c-a886-69541a3568e4 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:28.750336 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 145/293] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Jun 10 12:44:28 2022] GET /identity/v3/projects/485e375bd3394086be6c5cd49bd19e2d => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 12:44:28.756339 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66387f99-4b2d-4692-90c3-81d0b4239229 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:28.793698 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:28.878238 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66387f99-4b2d-4692-90c3-81d0b4239229 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:28.924785 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a076ece9-d8b9-4a7f-ac36-5eda430c44c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:28.925553 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a076ece9-d8b9-4a7f-ac36-5eda430c44c5 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:28.925845 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a076ece9-d8b9-4a7f-ac36-5eda430c44c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:28.926123 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a076ece9-d8b9-4a7f-ac36-5eda430c44c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:28.926971 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a076ece9-d8b9-4a7f-ac36-5eda430c44c5 None admin] RBAC: Adding query filter params (role.id=43106c664e1b40de984c0c62701149c8, scope.project.id=485e375bd3394086be6c5cd49bd19e2d, user.id=62cb9763157a443ab6b016f817628b2e) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:28.927284 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a076ece9-d8b9-4a7f-ac36-5eda430c44c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:28.928751 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a076ece9-d8b9-4a7f-ac36-5eda430c44c5 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:28.939165 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 149/294] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Jun 10 12:44:28 2022] GET /identity/v3/role_assignments?user.id=62cb9763157a443ab6b016f817628b2e&scope.project.id=485e375bd3394086be6c5cd49bd19e2d&role.id=43106c664e1b40de984c0c62701149c8 => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:29.539001 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f75c6f-5f14-4b3e-a165-ab0be33bb379 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:29.539507 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f75c6f-5f14-4b3e-a165-ab0be33bb379 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:29.539581 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f75c6f-5f14-4b3e-a165-ab0be33bb379 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:29.541740 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 146/295] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:44:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:44:29.546552 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7da9be-6ef9-45c1-8337-80e402bb243e None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:29.546940 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7da9be-6ef9-45c1-8337-80e402bb243e None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:29.547116 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7da9be-6ef9-45c1-8337-80e402bb243e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:29.564867 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-5e7da9be-6ef9-45c1-8337-80e402bb243e None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:44:29.566064 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5e7da9be-6ef9-45c1-8337-80e402bb243e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:44:29.635523 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 150/296] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:44:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 10 12:44:29.660207 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f75c6f-5f14-4b3e-a165-ab0be33bb379 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:29.723620 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83a92db-3699-4248-90d4-a0f14eb68225 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:29.723955 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83a92db-3699-4248-90d4-a0f14eb68225 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:29.724227 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83a92db-3699-4248-90d4-a0f14eb68225 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:29.725862 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 151/297] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:44:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:44:29.733366 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b830169-4804-420f-83ea-018c92b8db44 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:29.733771 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b830169-4804-420f-83ea-018c92b8db44 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:29.734157 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b830169-4804-420f-83ea-018c92b8db44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:29.751711 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-7b830169-4804-420f-83ea-018c92b8db44 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:44:29.753259 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-7b830169-4804-420f-83ea-018c92b8db44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:44:29.772050 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f75c6f-5f14-4b3e-a165-ab0be33bb379 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:29.798242 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:29.808834 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6aa833c-942a-46ba-b16d-2260b00814b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:29.809697 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aa833c-942a-46ba-b16d-2260b00814b7 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:29.809982 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aa833c-942a-46ba-b16d-2260b00814b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:29.810258 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aa833c-942a-46ba-b16d-2260b00814b7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:29.821266 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6aa833c-942a-46ba-b16d-2260b00814b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:44:29.826379 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 152/298] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:44:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 10 12:44:29.829322 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-a6aa833c-942a-46ba-b16d-2260b00814b7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 10 12:44:29.829322 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:44:29.829322 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:44:29.829322 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:44:29.829322 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:44:29.829322 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:44:29.829322 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:44:29.829322 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:44:29.829322 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:44:29.829322 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:44:29.829322 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:44:29.829322 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:44:29.829322 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:44:29.829322 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:44:29.829322 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:44:29.829322 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:44:29.830220 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:44:29.830220 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:44:29.830220 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:44:29.830220 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:44:29.830220 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:44:29.830220 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:44:29.830220 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:44:29.830220 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:44:29.830220 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:44:29.830220 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:44:29.830220 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:44:29.830220 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:44:29.830220 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:44:29.830220 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:44:29.830220 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:44:29.830220 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:44:29.831025 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:44:29.831025 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:44:29.831025 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:44:29.831025 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:44:29.831025 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:44:29.831025 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:44:29.831025 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:44:29.831025 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:44:29.831025 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:44:29.831025 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:44:29.831025 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 10 12:44:29.831025 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:44:29.831025 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 147/299] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Jun 10 12:44:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:44:29.840839 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b830169-4804-420f-83ea-018c92b8db44 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:29.861221 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6aa833c-942a-46ba-b16d-2260b00814b7 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:29.938647 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:29.957924 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b830169-4804-420f-83ea-018c92b8db44 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:29.994970 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6aa833c-942a-46ba-b16d-2260b00814b7 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:29.996834 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49029f70-e9ef-4405-8f8a-11e790000a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:29.997632 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49029f70-e9ef-4405-8f8a-11e790000a1a None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:29.997891 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49029f70-e9ef-4405-8f8a-11e790000a1a None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:29.998176 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49029f70-e9ef-4405-8f8a-11e790000a1a None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:29.999663 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49029f70-e9ef-4405-8f8a-11e790000a1a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:29.999663 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49029f70-e9ef-4405-8f8a-11e790000a1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:30.000781 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49029f70-e9ef-4405-8f8a-11e790000a1a None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:30.011553 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 153/300] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Jun 10 12:44:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:30.017368 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49029f70-e9ef-4405-8f8a-11e790000a1a None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:30.040532 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:30.045341 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4f92ff-f019-44b5-bd91-bd3384286038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:30.046554 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4f92ff-f019-44b5-bd91-bd3384286038 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:30.046930 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4f92ff-f019-44b5-bd91-bd3384286038 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:30.047267 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4f92ff-f019-44b5-bd91-bd3384286038 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:30.052266 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4f92ff-f019-44b5-bd91-bd3384286038 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:30.052266 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4f92ff-f019-44b5-bd91-bd3384286038 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:30.055928 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 148/301] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:44:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:30.059892 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4f92ff-f019-44b5-bd91-bd3384286038 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:30.140442 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49029f70-e9ef-4405-8f8a-11e790000a1a None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:30.185855 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7a815d-413e-45a1-8612-b37a2918a572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:30.186455 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4f92ff-f019-44b5-bd91-bd3384286038 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:30.187310 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7a815d-413e-45a1-8612-b37a2918a572 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:30.187906 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7a815d-413e-45a1-8612-b37a2918a572 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:30.188376 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7a815d-413e-45a1-8612-b37a2918a572 None admin] PATH_INFO: `/v3/users/86f547ea49dd496fa6970a314fd870aa` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:30.203576 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7a815d-413e-45a1-8612-b37a2918a572 None admin] RBAC: Authorizing `identity:get_user(user_id=86f547ea49dd496fa6970a314fd870aa)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:30.205077 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7a815d-413e-45a1-8612-b37a2918a572 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:30.206985 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 154/302] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Jun 10 12:44:30 2022] GET /identity/v3/users/86f547ea49dd496fa6970a314fd870aa => generated 293 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 12:44:30.208144 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4119067d-9def-457e-bef2-67385e38f841 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:30.208432 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4119067d-9def-457e-bef2-67385e38f841 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:30.208708 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4119067d-9def-457e-bef2-67385e38f841 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:30.211987 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 155/303] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:44:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:44:30.214666 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4119067d-9def-457e-bef2-67385e38f841 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:30.216777 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:30.227964 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6fc570-571a-45f5-8f8b-9bc96d90f0ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:30.229537 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6fc570-571a-45f5-8f8b-9bc96d90f0ac None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:30.229537 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6fc570-571a-45f5-8f8b-9bc96d90f0ac None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:30.229537 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6fc570-571a-45f5-8f8b-9bc96d90f0ac None admin] PATH_INFO: `/v3/groups` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:30.229975 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6fc570-571a-45f5-8f8b-9bc96d90f0ac None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:30.231311 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6fc570-571a-45f5-8f8b-9bc96d90f0ac None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:30.299486 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:30.307959 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 149/304] 10.222.0.27 () {60 vars in 1134 bytes} [Fri Jun 10 12:44:30 2022] POST /identity/v3/groups => generated 229 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 12:44:30.307959 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ca1409-a9bd-4029-b6ef-71b2c31b7f3e None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:30.307959 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ca1409-a9bd-4029-b6ef-71b2c31b7f3e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:30.307959 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ca1409-a9bd-4029-b6ef-71b2c31b7f3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:30.327069 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4119067d-9def-457e-bef2-67385e38f841 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:30.330917 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-d8ca1409-a9bd-4029-b6ef-71b2c31b7f3e None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:44:30.330917 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d8ca1409-a9bd-4029-b6ef-71b2c31b7f3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:44:30.377212 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75950830-5afd-461b-9d1d-89336bf01546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:30.378248 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75950830-5afd-461b-9d1d-89336bf01546 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:30.378718 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75950830-5afd-461b-9d1d-89336bf01546 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:30.379662 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75950830-5afd-461b-9d1d-89336bf01546 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:30.381081 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75950830-5afd-461b-9d1d-89336bf01546 None admin] RBAC: Adding query filter params (role.id=7d6c4b2adb9d43bca1a6573e8f7a9381, user.id=86f547ea49dd496fa6970a314fd870aa, scope.system=all) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:30.381734 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75950830-5afd-461b-9d1d-89336bf01546 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:30.383688 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75950830-5afd-461b-9d1d-89336bf01546 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:30.396770 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 156/305] 10.222.0.27 () {56 vars in 1308 bytes} [Fri Jun 10 12:44:30 2022] GET /identity/v3/role_assignments?user.id=86f547ea49dd496fa6970a314fd870aa&scope.system=all&role.id=7d6c4b2adb9d43bca1a6573e8f7a9381 => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:30.414883 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 150/306] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:44:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 10 12:44:30.438398 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75950830-5afd-461b-9d1d-89336bf01546 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:30.555387 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75950830-5afd-461b-9d1d-89336bf01546 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:30.592498 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715bfc08-5c19-4bb5-b274-9e56017fd70e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:30.593216 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715bfc08-5c19-4bb5-b274-9e56017fd70e None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:30.593496 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715bfc08-5c19-4bb5-b274-9e56017fd70e None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:30.593770 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715bfc08-5c19-4bb5-b274-9e56017fd70e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:30.604574 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715bfc08-5c19-4bb5-b274-9e56017fd70e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:44:30.612333 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-715bfc08-5c19-4bb5-b274-9e56017fd70e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 10 12:44:30.612333 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:44:30.612333 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:44:30.612333 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:44:30.612333 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:44:30.612333 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:44:30.612333 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:44:30.612333 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:44:30.612333 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:44:30.612333 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:44:30.612333 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:44:30.612333 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:44:30.612333 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:44:30.612333 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:44:30.612333 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:44:30.612333 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:44:30.613319 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:44:30.613319 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:44:30.613319 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:44:30.613319 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:44:30.613319 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:44:30.613319 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:44:30.613319 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:44:30.613319 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:44:30.613319 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:44:30.613319 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:44:30.613319 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:44:30.613319 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:44:30.613319 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:44:30.613319 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:44:30.613319 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:44:30.613319 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:44:30.614134 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:44:30.614134 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:44:30.614134 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:44:30.614134 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:44:30.614134 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:44:30.614134 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:44:30.614134 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:44:30.614134 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:44:30.614134 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:44:30.614134 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:44:30.614134 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 10 12:44:30.614134 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:44:30.618439 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 157/307] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Jun 10 12:44:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:44:30.620431 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ca1409-a9bd-4029-b6ef-71b2c31b7f3e None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:30.723744 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ca1409-a9bd-4029-b6ef-71b2c31b7f3e None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:30.737930 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:30.759429 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a375c5bb-08d8-43a9-95a5-db16aa409ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:30.769971 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a375c5bb-08d8-43a9-95a5-db16aa409ff6 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:30.769971 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a375c5bb-08d8-43a9-95a5-db16aa409ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:30.769971 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a375c5bb-08d8-43a9-95a5-db16aa409ff6 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:30.769971 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a375c5bb-08d8-43a9-95a5-db16aa409ff6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:30.769971 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a375c5bb-08d8-43a9-95a5-db16aa409ff6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:30.769971 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a375c5bb-08d8-43a9-95a5-db16aa409ff6 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:30.772298 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 151/308] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Jun 10 12:44:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 12:44:30.777792 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715bfc08-5c19-4bb5-b274-9e56017fd70e None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:30.780855 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:30.913981 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715bfc08-5c19-4bb5-b274-9e56017fd70e None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:30.956237 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181a2f15-b24b-4811-ad7a-7087edd0fac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:30.957441 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181a2f15-b24b-4811-ad7a-7087edd0fac5 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:30.957861 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181a2f15-b24b-4811-ad7a-7087edd0fac5 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:30.958267 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181a2f15-b24b-4811-ad7a-7087edd0fac5 None admin] PATH_INFO: `/v3/users/1ac03230095448b2bd230560807af974` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:30.959593 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181a2f15-b24b-4811-ad7a-7087edd0fac5 None admin] RBAC: Authorizing `identity:get_user(user_id=1ac03230095448b2bd230560807af974)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:30.961282 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181a2f15-b24b-4811-ad7a-7087edd0fac5 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:30.963517 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 158/309] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Jun 10 12:44:30 2022] GET /identity/v3/users/1ac03230095448b2bd230560807af974 => generated 293 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:30.968913 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a375c5bb-08d8-43a9-95a5-db16aa409ff6 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:31.119073 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a375c5bb-08d8-43a9-95a5-db16aa409ff6 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:31.168142 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0954511d-eea0-4f78-b2b6-bb436f461fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:31.169063 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0954511d-eea0-4f78-b2b6-bb436f461fce None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:31.169346 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0954511d-eea0-4f78-b2b6-bb436f461fce None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:31.169629 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0954511d-eea0-4f78-b2b6-bb436f461fce None admin] PATH_INFO: `/v3/system/users/1ac03230095448b2bd230560807af974/roles/43106c664e1b40de984c0c62701149c8` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:31.174330 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0954511d-eea0-4f78-b2b6-bb436f461fce None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=1ac03230095448b2bd230560807af974, role_id=43106c664e1b40de984c0c62701149c8)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:31.175770 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0954511d-eea0-4f78-b2b6-bb436f461fce None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:31.249415 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 152/310] 10.222.0.27 () {58 vars in 1334 bytes} [Fri Jun 10 12:44:30 2022] PUT /identity/v3/system/users/1ac03230095448b2bd230560807af974/roles/43106c664e1b40de984c0c62701149c8 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 12:44:31.257520 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04be77d-be72-4563-823e-1648246b9d89 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:31.257520 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04be77d-be72-4563-823e-1648246b9d89 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:31.257520 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04be77d-be72-4563-823e-1648246b9d89 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:31.259538 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 159/311] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:44:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:44:31.265022 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d05ad4-92ed-44e1-b4db-dd3754f69c13 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:31.265333 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d05ad4-92ed-44e1-b4db-dd3754f69c13 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:31.265602 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d05ad4-92ed-44e1-b4db-dd3754f69c13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:31.292487 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-58d05ad4-92ed-44e1-b4db-dd3754f69c13 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:44:31.294524 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-58d05ad4-92ed-44e1-b4db-dd3754f69c13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:44:31.352270 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:31.399261 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 153/312] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:44:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 10 12:44:31.424869 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04be77d-be72-4563-823e-1648246b9d89 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:31.526188 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:31.563387 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04be77d-be72-4563-823e-1648246b9d89 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:31.602496 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1858aafb-559c-4063-9d59-93dca631ae49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:31.603755 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1858aafb-559c-4063-9d59-93dca631ae49 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:31.604284 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1858aafb-559c-4063-9d59-93dca631ae49 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:31.604782 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1858aafb-559c-4063-9d59-93dca631ae49 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:31.617922 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1858aafb-559c-4063-9d59-93dca631ae49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:44:31.626349 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-1858aafb-559c-4063-9d59-93dca631ae49 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 10 12:44:31.626349 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:44:31.626349 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:44:31.626349 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:44:31.626349 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:44:31.626349 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:44:31.626349 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:44:31.626349 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:44:31.626349 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:44:31.626349 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:44:31.626349 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:44:31.626349 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:44:31.626349 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:44:31.626349 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:44:31.626349 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:44:31.626349 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:44:31.627332 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:44:31.627332 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:44:31.627332 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:44:31.627332 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:44:31.627332 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:44:31.627332 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:44:31.627332 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:44:31.627332 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:44:31.627332 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:44:31.627332 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:44:31.627332 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:44:31.627332 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:44:31.627332 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:44:31.627332 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:44:31.627332 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:44:31.627332 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:44:31.628166 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:44:31.628166 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:44:31.628166 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:44:31.628166 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:44:31.628166 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:44:31.628166 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:44:31.628166 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:44:31.628166 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:44:31.628166 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:44:31.628166 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:44:31.628166 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 10 12:44:31.628166 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:44:31.630896 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 160/313] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Jun 10 12:44:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:44:31.637521 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d05ad4-92ed-44e1-b4db-dd3754f69c13 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:31.739020 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d05ad4-92ed-44e1-b4db-dd3754f69c13 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:31.772953 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60db656-b964-4af4-b6e8-5c0fd428b852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:31.773663 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60db656-b964-4af4-b6e8-5c0fd428b852 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:31.773951 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60db656-b964-4af4-b6e8-5c0fd428b852 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:31.774225 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60db656-b964-4af4-b6e8-5c0fd428b852 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:31.774967 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60db656-b964-4af4-b6e8-5c0fd428b852 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:31.775894 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60db656-b964-4af4-b6e8-5c0fd428b852 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:31.777338 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60db656-b964-4af4-b6e8-5c0fd428b852 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:31.785616 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 154/314] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Jun 10 12:44:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 12:44:31.791332 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1858aafb-559c-4063-9d59-93dca631ae49 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:31.897542 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1858aafb-559c-4063-9d59-93dca631ae49 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:31.932130 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d1c9eb-7a31-4567-b95d-31c9058136cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:31.933222 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:31.933687 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d1c9eb-7a31-4567-b95d-31c9058136cf None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:31.934367 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d1c9eb-7a31-4567-b95d-31c9058136cf None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:31.934906 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d1c9eb-7a31-4567-b95d-31c9058136cf None admin] PATH_INFO: `/v3/users/ef91f0d46059481db5f9a5a0eb635aae` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:31.936373 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d1c9eb-7a31-4567-b95d-31c9058136cf None admin] RBAC: Authorizing `identity:get_user(user_id=ef91f0d46059481db5f9a5a0eb635aae)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:31.938011 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d1c9eb-7a31-4567-b95d-31c9058136cf None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:31.940144 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 161/315] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Jun 10 12:44:31 2022] GET /identity/v3/users/ef91f0d46059481db5f9a5a0eb635aae => generated 297 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:31.945840 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60db656-b964-4af4-b6e8-5c0fd428b852 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:31.977724 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:32.049082 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60db656-b964-4af4-b6e8-5c0fd428b852 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:32.085164 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e60d78-e4aa-4eb5-a869-fee8fdcdcecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:32.086150 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e60d78-e4aa-4eb5-a869-fee8fdcdcecf None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:32.086617 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e60d78-e4aa-4eb5-a869-fee8fdcdcecf None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:32.087095 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e60d78-e4aa-4eb5-a869-fee8fdcdcecf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:32.162825 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e60d78-e4aa-4eb5-a869-fee8fdcdcecf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:32.164665 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e60d78-e4aa-4eb5-a869-fee8fdcdcecf None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:32.196763 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:32.266832 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 155/316] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Jun 10 12:44:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 10 12:44:32.272643 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d1c9eb-7a31-4567-b95d-31c9058136cf None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:32.380696 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d1c9eb-7a31-4567-b95d-31c9058136cf None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:32.415064 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d635618b-fd10-416b-9151-4cfbfb36f3df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:32.416846 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d635618b-fd10-416b-9151-4cfbfb36f3df None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:32.417400 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d635618b-fd10-416b-9151-4cfbfb36f3df None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:32.417893 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d635618b-fd10-416b-9151-4cfbfb36f3df None admin] PATH_INFO: `/v3/projects/485e375bd3394086be6c5cd49bd19e2d` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:32.419301 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d635618b-fd10-416b-9151-4cfbfb36f3df None admin] RBAC: Authorizing `identity:get_project(project_id=485e375bd3394086be6c5cd49bd19e2d)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:32.421047 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d635618b-fd10-416b-9151-4cfbfb36f3df None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:32.423238 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 162/317] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Jun 10 12:44:32 2022] GET /identity/v3/projects/485e375bd3394086be6c5cd49bd19e2d => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:32.429121 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e60d78-e4aa-4eb5-a869-fee8fdcdcecf None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:32.533707 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e60d78-e4aa-4eb5-a869-fee8fdcdcecf None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:32.564004 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:32.569664 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91227bc7-00b5-4747-86da-6bce0b3f76ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:32.570337 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91227bc7-00b5-4747-86da-6bce0b3f76ba None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:32.570620 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91227bc7-00b5-4747-86da-6bce0b3f76ba None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:32.570895 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91227bc7-00b5-4747-86da-6bce0b3f76ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:32.571779 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91227bc7-00b5-4747-86da-6bce0b3f76ba None admin] RBAC: Adding query filter params (role.id=7d6c4b2adb9d43bca1a6573e8f7a9381, scope.project.id=485e375bd3394086be6c5cd49bd19e2d, user.id=ef91f0d46059481db5f9a5a0eb635aae) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:32.572154 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91227bc7-00b5-4747-86da-6bce0b3f76ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:32.573661 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91227bc7-00b5-4747-86da-6bce0b3f76ba None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:32.580113 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 156/318] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Jun 10 12:44:32 2022] GET /identity/v3/role_assignments?user.id=ef91f0d46059481db5f9a5a0eb635aae&scope.project.id=485e375bd3394086be6c5cd49bd19e2d&role.id=7d6c4b2adb9d43bca1a6573e8f7a9381 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:32.674533 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39827d71-4730-4c73-86c4-f9659e893a3b None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:32.675127 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39827d71-4730-4c73-86c4-f9659e893a3b None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:32.675127 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39827d71-4730-4c73-86c4-f9659e893a3b None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:32.676788 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 163/319] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:44:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:44:32.682755 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c896a77-961c-4817-88ef-2264bdf1374b None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:32.683259 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c896a77-961c-4817-88ef-2264bdf1374b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:32.683365 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c896a77-961c-4817-88ef-2264bdf1374b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:32.700802 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-5c896a77-961c-4817-88ef-2264bdf1374b None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:44:32.702005 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5c896a77-961c-4817-88ef-2264bdf1374b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:44:32.735819 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d9f822-6990-417c-971d-3d24b69b79a0 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:32.736174 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d9f822-6990-417c-971d-3d24b69b79a0 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:32.736419 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d9f822-6990-417c-971d-3d24b69b79a0 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:32.737927 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 164/320] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:44:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 10 12:44:32.745728 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a89e632-de05-40d6-836c-c2ac5267f303 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:32.746016 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a89e632-de05-40d6-836c-c2ac5267f303 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:32.746299 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a89e632-de05-40d6-836c-c2ac5267f303 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:32.767695 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-5a89e632-de05-40d6-836c-c2ac5267f303 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:44:32.769036 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5a89e632-de05-40d6-836c-c2ac5267f303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:44:32.780414 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 157/321] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:44:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 10 12:44:32.803148 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c896a77-961c-4817-88ef-2264bdf1374b None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:32.840135 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:32.851393 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 165/322] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:44:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 10 12:44:32.879659 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a89e632-de05-40d6-836c-c2ac5267f303 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:32.913686 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c896a77-961c-4817-88ef-2264bdf1374b None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:32.937153 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:32.947342 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5a6dd9-e8a0-4aef-ba57-d35f093835a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:32.948191 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5a6dd9-e8a0-4aef-ba57-d35f093835a6 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:32.948470 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5a6dd9-e8a0-4aef-ba57-d35f093835a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:32.948751 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5a6dd9-e8a0-4aef-ba57-d35f093835a6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:32.958740 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5a6dd9-e8a0-4aef-ba57-d35f093835a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:44:32.967111 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-ee5a6dd9-e8a0-4aef-ba57-d35f093835a6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 10 12:44:32.967111 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:44:32.967111 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:44:32.967111 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:44:32.967111 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:44:32.967111 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:44:32.967111 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:44:32.967111 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:44:32.967111 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:44:32.967111 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:44:32.967111 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:44:32.967111 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:44:32.967111 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:44:32.967111 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:44:32.967111 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:44:32.967111 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:44:32.968166 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:44:32.968166 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:44:32.968166 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:44:32.968166 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:44:32.968166 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:44:32.968166 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:44:32.968166 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:44:32.968166 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:44:32.968166 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:44:32.968166 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:44:32.968166 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:44:32.968166 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:44:32.968166 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:44:32.968166 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:44:32.968166 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:44:32.968166 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:44:32.968987 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:44:32.968987 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:44:32.968987 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:44:32.968987 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:44:32.968987 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:44:32.968987 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:44:32.968987 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:44:32.968987 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:44:32.968987 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:44:32.968987 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:44:32.968987 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 10 12:44:32.968987 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:44:32.968987 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 158/323] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Jun 10 12:44:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 10 12:44:32.975883 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5a6dd9-e8a0-4aef-ba57-d35f093835a6 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:32.994426 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a89e632-de05-40d6-836c-c2ac5267f303 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:33.034303 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7dcdd6-ae44-41e4-bbf8-1000527f543d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:33.035095 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7dcdd6-ae44-41e4-bbf8-1000527f543d None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:33.035378 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7dcdd6-ae44-41e4-bbf8-1000527f543d None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:33.035652 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7dcdd6-ae44-41e4-bbf8-1000527f543d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:33.037346 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7dcdd6-ae44-41e4-bbf8-1000527f543d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:33.038859 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7dcdd6-ae44-41e4-bbf8-1000527f543d None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:33.040944 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 166/324] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:44:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:33.046224 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7dcdd6-ae44-41e4-bbf8-1000527f543d None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:33.079478 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5a6dd9-e8a0-4aef-ba57-d35f093835a6 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:33.109081 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:33.114849 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e52dfcf-0135-4608-9f9b-92a95b0da28d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:33.115540 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e52dfcf-0135-4608-9f9b-92a95b0da28d None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:33.119422 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e52dfcf-0135-4608-9f9b-92a95b0da28d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:33.119422 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e52dfcf-0135-4608-9f9b-92a95b0da28d None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:33.119422 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e52dfcf-0135-4608-9f9b-92a95b0da28d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:33.119422 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e52dfcf-0135-4608-9f9b-92a95b0da28d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:33.119422 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e52dfcf-0135-4608-9f9b-92a95b0da28d None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:33.131499 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 159/325] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Jun 10 12:44:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 12:44:33.132801 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e52dfcf-0135-4608-9f9b-92a95b0da28d None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:33.147573 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7dcdd6-ae44-41e4-bbf8-1000527f543d None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:33.185122 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9357ee4a-8aa5-4378-bf2e-6ca07441ec74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:33.186010 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9357ee4a-8aa5-4378-bf2e-6ca07441ec74 None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:33.186470 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9357ee4a-8aa5-4378-bf2e-6ca07441ec74 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:33.186891 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9357ee4a-8aa5-4378-bf2e-6ca07441ec74 None admin] PATH_INFO: `/v3/groups` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:33.187794 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9357ee4a-8aa5-4378-bf2e-6ca07441ec74 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:33.188310 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:33.189215 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9357ee4a-8aa5-4378-bf2e-6ca07441ec74 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:33.242885 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e52dfcf-0135-4608-9f9b-92a95b0da28d None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:33.279940 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 167/326] 10.222.0.27 () {60 vars in 1134 bytes} [Fri Jun 10 12:44:33 2022] POST /identity/v3/groups => generated 226 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 12:44:33.286699 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07fa7cf-24eb-4a74-803c-a772fcf2cbc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:33.288039 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07fa7cf-24eb-4a74-803c-a772fcf2cbc7 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:33.288396 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07fa7cf-24eb-4a74-803c-a772fcf2cbc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:33.288770 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07fa7cf-24eb-4a74-803c-a772fcf2cbc7 None admin] PATH_INFO: `/v3/users/86f547ea49dd496fa6970a314fd870aa` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:33.290219 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07fa7cf-24eb-4a74-803c-a772fcf2cbc7 None admin] RBAC: Authorizing `identity:get_user(user_id=86f547ea49dd496fa6970a314fd870aa)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:33.291776 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07fa7cf-24eb-4a74-803c-a772fcf2cbc7 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:33.293772 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 160/327] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Jun 10 12:44:33 2022] GET /identity/v3/users/86f547ea49dd496fa6970a314fd870aa => generated 293 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:33.299416 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9357ee4a-8aa5-4378-bf2e-6ca07441ec74 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:33.409263 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9357ee4a-8aa5-4378-bf2e-6ca07441ec74 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:33.434583 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:33.447704 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27950a0-d9bb-4151-aad3-38a51e6dafd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:33.447704 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27950a0-d9bb-4151-aad3-38a51e6dafd8 None admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:33.447704 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27950a0-d9bb-4151-aad3-38a51e6dafd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:33.447704 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27950a0-d9bb-4151-aad3-38a51e6dafd8 None admin] PATH_INFO: `/v3/system/users/86f547ea49dd496fa6970a314fd870aa/roles/7d6c4b2adb9d43bca1a6573e8f7a9381` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:33.449076 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27950a0-d9bb-4151-aad3-38a51e6dafd8 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=86f547ea49dd496fa6970a314fd870aa, role_id=7d6c4b2adb9d43bca1a6573e8f7a9381)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:33.450461 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27950a0-d9bb-4151-aad3-38a51e6dafd8 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:33.512744 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 168/328] 10.222.0.27 () {58 vars in 1334 bytes} [Fri Jun 10 12:44:33 2022] PUT /identity/v3/system/users/86f547ea49dd496fa6970a314fd870aa/roles/7d6c4b2adb9d43bca1a6573e8f7a9381 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 10 12:44:33.550244 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f199b04-94ff-4d2e-8a63-89aaa9c2e0d4 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:33.550570 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f199b04-94ff-4d2e-8a63-89aaa9c2e0d4 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:33.550820 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f199b04-94ff-4d2e-8a63-89aaa9c2e0d4 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:33.552328 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 161/329] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:44:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:44:33.558998 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6257cec-1f25-4507-97a8-50002d13e7e0 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:33.559287 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6257cec-1f25-4507-97a8-50002d13e7e0 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:33.559640 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6257cec-1f25-4507-97a8-50002d13e7e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:33.579565 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-b6257cec-1f25-4507-97a8-50002d13e7e0 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:44:33.580922 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b6257cec-1f25-4507-97a8-50002d13e7e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:44:33.666803 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 169/330] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:44:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 10 12:44:33.685862 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f199b04-94ff-4d2e-8a63-89aaa9c2e0d4 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:33.798364 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f199b04-94ff-4d2e-8a63-89aaa9c2e0d4 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:33.814091 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:33.834621 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650e1b05-7ee5-44ce-9ccb-0f4c00366606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:33.835425 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650e1b05-7ee5-44ce-9ccb-0f4c00366606 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:33.835707 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650e1b05-7ee5-44ce-9ccb-0f4c00366606 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:33.836013 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650e1b05-7ee5-44ce-9ccb-0f4c00366606 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:33.847009 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650e1b05-7ee5-44ce-9ccb-0f4c00366606 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:44:33.855467 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-650e1b05-7ee5-44ce-9ccb-0f4c00366606 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 10 12:44:33.855467 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:44:33.855467 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:44:33.855467 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:44:33.855467 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:44:33.855467 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:44:33.855467 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:44:33.855467 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:44:33.855467 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:44:33.855467 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:44:33.855467 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:44:33.855467 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:44:33.855467 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:44:33.855467 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:44:33.855467 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:44:33.855467 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:44:33.856471 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:44:33.856471 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:44:33.856471 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:44:33.856471 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:44:33.856471 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:44:33.856471 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:44:33.856471 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:44:33.856471 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:44:33.856471 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:44:33.856471 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:44:33.856471 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:44:33.856471 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:44:33.856471 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:44:33.856471 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:44:33.856471 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:44:33.856471 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:44:33.857271 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:44:33.857271 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:44:33.857271 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:44:33.857271 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:44:33.857271 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:44:33.857271 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:44:33.857271 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:44:33.857271 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:44:33.857271 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:44:33.857271 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:44:33.857271 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 10 12:44:33.857271 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:44:33.857271 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 162/331] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Jun 10 12:44:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:44:33.863879 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6257cec-1f25-4507-97a8-50002d13e7e0 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:33.974432 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6257cec-1f25-4507-97a8-50002d13e7e0 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:33.994832 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:34.012571 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a287cb0a-bda8-4764-a243-5adec6fac4a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:34.013633 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a287cb0a-bda8-4764-a243-5adec6fac4a0 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:34.014155 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a287cb0a-bda8-4764-a243-5adec6fac4a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:34.014640 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a287cb0a-bda8-4764-a243-5adec6fac4a0 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:34.015637 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a287cb0a-bda8-4764-a243-5adec6fac4a0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:34.016164 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a287cb0a-bda8-4764-a243-5adec6fac4a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:34.017740 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a287cb0a-bda8-4764-a243-5adec6fac4a0 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:34.026949 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 170/332] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Jun 10 12:44:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:34.032347 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650e1b05-7ee5-44ce-9ccb-0f4c00366606 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:34.159529 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650e1b05-7ee5-44ce-9ccb-0f4c00366606 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:34.204661 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe132f51-bfab-4639-9ec8-8a4ca0e48de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:34.205557 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe132f51-bfab-4639-9ec8-8a4ca0e48de8 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:34.205832 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe132f51-bfab-4639-9ec8-8a4ca0e48de8 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:34.206108 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe132f51-bfab-4639-9ec8-8a4ca0e48de8 None admin] PATH_INFO: `/v3/users/1ac03230095448b2bd230560807af974` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:34.208243 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe132f51-bfab-4639-9ec8-8a4ca0e48de8 None admin] RBAC: Authorizing `identity:get_user(user_id=1ac03230095448b2bd230560807af974)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:34.209703 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe132f51-bfab-4639-9ec8-8a4ca0e48de8 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:34.211336 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 163/333] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Jun 10 12:44:34 2022] GET /identity/v3/users/1ac03230095448b2bd230560807af974 => generated 293 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:34.217719 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a287cb0a-bda8-4764-a243-5adec6fac4a0 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:34.339474 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a287cb0a-bda8-4764-a243-5adec6fac4a0 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:34.376075 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d704d8f-d7e4-4f08-b499-f68ebca9773e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:34.376733 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d704d8f-d7e4-4f08-b499-f68ebca9773e None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:34.377021 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d704d8f-d7e4-4f08-b499-f68ebca9773e None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:34.377337 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d704d8f-d7e4-4f08-b499-f68ebca9773e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:34.378177 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d704d8f-d7e4-4f08-b499-f68ebca9773e None admin] RBAC: Adding query filter params (role.id=43106c664e1b40de984c0c62701149c8, user.id=1ac03230095448b2bd230560807af974, scope.system=all) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:34.378489 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d704d8f-d7e4-4f08-b499-f68ebca9773e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:34.379931 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d704d8f-d7e4-4f08-b499-f68ebca9773e None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:34.391979 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 171/334] 10.222.0.27 () {56 vars in 1308 bytes} [Fri Jun 10 12:44:34 2022] GET /identity/v3/role_assignments?user.id=1ac03230095448b2bd230560807af974&scope.system=all&role.id=43106c664e1b40de984c0c62701149c8 => generated 510 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:35.880399 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e480018-7157-41a5-afe0-b890cafc1cbe None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:35.880880 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e480018-7157-41a5-afe0-b890cafc1cbe None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:35.880955 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e480018-7157-41a5-afe0-b890cafc1cbe None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:35.882525 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 164/335] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:44:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:44:35.888365 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b9eb31-6d47-400a-880d-4be9ab1fc94b None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:35.888761 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b9eb31-6d47-400a-880d-4be9ab1fc94b None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:35.888926 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b9eb31-6d47-400a-880d-4be9ab1fc94b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:35.909056 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-26b9eb31-6d47-400a-880d-4be9ab1fc94b None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:44:35.910249 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-26b9eb31-6d47-400a-880d-4be9ab1fc94b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:44:35.954024 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:35.996498 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 172/336] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:44:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 10 12:44:36.031747 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e480018-7157-41a5-afe0-b890cafc1cbe None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:36.093357 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:36.109414 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c010b9-01b7-4a80-9e48-0609f005f401 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:36.109702 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c010b9-01b7-4a80-9e48-0609f005f401 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:36.109983 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c010b9-01b7-4a80-9e48-0609f005f401 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:36.111355 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 173/337] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:44:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:44:36.116521 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca30780-a44c-4bbe-bb2e-473d099b694a None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:36.116744 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca30780-a44c-4bbe-bb2e-473d099b694a None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:36.117032 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca30780-a44c-4bbe-bb2e-473d099b694a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:36.145381 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-eca30780-a44c-4bbe-bb2e-473d099b694a None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:44:36.146968 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-eca30780-a44c-4bbe-bb2e-473d099b694a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:44:36.222154 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e480018-7157-41a5-afe0-b890cafc1cbe None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:36.254237 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 174/338] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:44:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 10 12:44:36.296044 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca30780-a44c-4bbe-bb2e-473d099b694a None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:36.309784 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1189e2d3-f952-4a4c-9afb-2d76ef77ae9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:36.310607 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1189e2d3-f952-4a4c-9afb-2d76ef77ae9f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:36.310889 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1189e2d3-f952-4a4c-9afb-2d76ef77ae9f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:36.311164 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1189e2d3-f952-4a4c-9afb-2d76ef77ae9f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:36.356823 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1189e2d3-f952-4a4c-9afb-2d76ef77ae9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:44:36.372735 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-1189e2d3-f952-4a4c-9afb-2d76ef77ae9f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 10 12:44:36.372735 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:44:36.372735 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:44:36.372735 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:44:36.372735 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:44:36.372735 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:44:36.372735 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:44:36.372735 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:44:36.372735 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:44:36.372735 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:44:36.372735 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:44:36.372735 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:44:36.372735 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:44:36.372735 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:44:36.372735 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:44:36.372735 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:44:36.373650 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:44:36.373650 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:44:36.373650 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:44:36.373650 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:44:36.373650 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:44:36.373650 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:44:36.373650 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:44:36.373650 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:44:36.373650 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:44:36.373650 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:44:36.373650 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:44:36.373650 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:44:36.373650 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:44:36.373650 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:44:36.373650 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:44:36.373650 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:44:36.374886 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:44:36.374886 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:44:36.374886 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:44:36.374886 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:44:36.374886 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:44:36.374886 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:44:36.374886 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:44:36.374886 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:44:36.374886 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:44:36.374886 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:44:36.374886 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 10 12:44:36.374886 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:44:36.374886 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 165/339] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Jun 10 12:44:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:44:36.380844 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1189e2d3-f952-4a4c-9afb-2d76ef77ae9f None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:36.443182 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca30780-a44c-4bbe-bb2e-473d099b694a None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:36.454059 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:36.498690 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1189e2d3-f952-4a4c-9afb-2d76ef77ae9f None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:36.500258 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:36.508209 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ae08d0-9214-467b-a2e2-4707005d1015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:36.509046 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ae08d0-9214-467b-a2e2-4707005d1015 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:36.509370 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ae08d0-9214-467b-a2e2-4707005d1015 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:36.509705 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ae08d0-9214-467b-a2e2-4707005d1015 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:36.529838 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ae08d0-9214-467b-a2e2-4707005d1015 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:44:36.541935 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-58ae08d0-9214-467b-a2e2-4707005d1015 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 10 12:44:36.541935 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:44:36.541935 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:44:36.541935 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:44:36.541935 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:44:36.541935 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:44:36.541935 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:44:36.541935 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:44:36.541935 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:44:36.541935 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:44:36.541935 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:44:36.541935 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:44:36.541935 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:44:36.541935 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:44:36.541935 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:44:36.541935 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:44:36.542883 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:44:36.542883 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:44:36.542883 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:44:36.542883 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:44:36.542883 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:44:36.542883 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:44:36.542883 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:44:36.542883 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:44:36.542883 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:44:36.542883 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:44:36.542883 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:44:36.542883 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:44:36.542883 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:44:36.542883 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:44:36.542883 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:44:36.542883 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:44:36.543685 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:44:36.543685 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:44:36.543685 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:44:36.543685 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:44:36.543685 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:44:36.543685 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:44:36.543685 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:44:36.543685 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:44:36.543685 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:44:36.543685 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:44:36.543685 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 10 12:44:36.543685 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:44:36.545051 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 175/340] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Jun 10 12:44:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:44:36.549125 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9396529b-2cfa-4e42-aa08-a2c12063bd6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:36.549313 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b47f9c3-1309-4d6e-8733-7a456ae88913 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:36.549400 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9396529b-2cfa-4e42-aa08-a2c12063bd6b None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:36.549470 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b47f9c3-1309-4d6e-8733-7a456ae88913 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:36.549565 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9396529b-2cfa-4e42-aa08-a2c12063bd6b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:36.549643 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b47f9c3-1309-4d6e-8733-7a456ae88913 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:36.549718 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9396529b-2cfa-4e42-aa08-a2c12063bd6b None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:36.549718 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9396529b-2cfa-4e42-aa08-a2c12063bd6b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:36.549718 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9396529b-2cfa-4e42-aa08-a2c12063bd6b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:36.549718 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9396529b-2cfa-4e42-aa08-a2c12063bd6b None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:36.556241 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 176/341] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:44:36 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:44:36.557583 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b47f9c3-1309-4d6e-8733-7a456ae88913 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:36.568227 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 166/342] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Jun 10 12:44:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:36.569575 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363f418f-0ab3-4965-bf0e-c496cedc79d5 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:36.569866 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363f418f-0ab3-4965-bf0e-c496cedc79d5 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:36.570144 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363f418f-0ab3-4965-bf0e-c496cedc79d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:36.602714 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-363f418f-0ab3-4965-bf0e-c496cedc79d5 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:44:36.606978 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-363f418f-0ab3-4965-bf0e-c496cedc79d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:44:36.718522 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b47f9c3-1309-4d6e-8733-7a456ae88913 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:36.765073 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 167/343] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:44:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 10 12:44:36.765958 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363f418f-0ab3-4965-bf0e-c496cedc79d5 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:36.801258 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbaf5e8-c860-4649-8ed5-126cb7f11da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:36.802301 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbaf5e8-c860-4649-8ed5-126cb7f11da9 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:36.802780 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbaf5e8-c860-4649-8ed5-126cb7f11da9 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:36.803247 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbaf5e8-c860-4649-8ed5-126cb7f11da9 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:36.804218 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbaf5e8-c860-4649-8ed5-126cb7f11da9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:36.804739 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbaf5e8-c860-4649-8ed5-126cb7f11da9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:36.806357 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbaf5e8-c860-4649-8ed5-126cb7f11da9 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:36.830001 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 177/344] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Jun 10 12:44:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:36.831247 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbaf5e8-c860-4649-8ed5-126cb7f11da9 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:36.888615 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:36.891512 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:36.899297 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363f418f-0ab3-4965-bf0e-c496cedc79d5 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:36.944582 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f135a8-3c5f-456f-8141-fd0c1fff3d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:36.945464 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f135a8-3c5f-456f-8141-fd0c1fff3d22 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:36.945743 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f135a8-3c5f-456f-8141-fd0c1fff3d22 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:36.946015 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f135a8-3c5f-456f-8141-fd0c1fff3d22 None admin] PATH_INFO: `/v3/users/86f547ea49dd496fa6970a314fd870aa` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:36.949816 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f135a8-3c5f-456f-8141-fd0c1fff3d22 None admin] RBAC: Authorizing `identity:get_user(user_id=86f547ea49dd496fa6970a314fd870aa)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:36.951254 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f135a8-3c5f-456f-8141-fd0c1fff3d22 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:36.961761 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 168/345] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Jun 10 12:44:36 2022] GET /identity/v3/users/86f547ea49dd496fa6970a314fd870aa => generated 293 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:36.963238 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f135a8-3c5f-456f-8141-fd0c1fff3d22 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:37.084433 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbaf5e8-c860-4649-8ed5-126cb7f11da9 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:37.128871 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e996bf14-e661-4edb-8146-c81fc8c48d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:37.129520 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f135a8-3c5f-456f-8141-fd0c1fff3d22 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:37.130291 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e996bf14-e661-4edb-8146-c81fc8c48d2f None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:37.130789 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e996bf14-e661-4edb-8146-c81fc8c48d2f None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:37.131264 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e996bf14-e661-4edb-8146-c81fc8c48d2f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:37.144159 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e996bf14-e661-4edb-8146-c81fc8c48d2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:44:37.152985 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-e996bf14-e661-4edb-8146-c81fc8c48d2f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 10 12:44:37.152985 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:44:37.152985 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:44:37.152985 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:44:37.152985 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:44:37.152985 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:44:37.152985 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:44:37.152985 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:44:37.152985 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:44:37.152985 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:44:37.152985 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:44:37.152985 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:44:37.152985 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:44:37.152985 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:44:37.152985 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:44:37.152985 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:44:37.154163 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:44:37.154163 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:44:37.154163 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:44:37.154163 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:44:37.154163 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:44:37.154163 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:44:37.154163 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:44:37.154163 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:44:37.154163 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:44:37.154163 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:44:37.154163 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:44:37.154163 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:44:37.154163 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:44:37.154163 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:44:37.154163 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:44:37.154163 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:44:37.155077 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:44:37.155077 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:44:37.155077 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:44:37.155077 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:44:37.155077 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:44:37.155077 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:44:37.155077 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:44:37.155077 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:44:37.155077 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:44:37.155077 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:44:37.155077 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 10 12:44:37.155077 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:44:37.157478 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 178/346] 10.222.0.27 () {56 vars in 1114 bytes} [Fri Jun 10 12:44:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:44:37.158583 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e996bf14-e661-4edb-8146-c81fc8c48d2f None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:37.176544 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82328ad3-7059-4f04-af6a-5be8f636986d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:37.177743 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82328ad3-7059-4f04-af6a-5be8f636986d None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:37.178177 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82328ad3-7059-4f04-af6a-5be8f636986d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:37.178587 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82328ad3-7059-4f04-af6a-5be8f636986d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:37.215617 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:37.272124 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82328ad3-7059-4f04-af6a-5be8f636986d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:37.273710 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:37.275524 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82328ad3-7059-4f04-af6a-5be8f636986d None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:37.293893 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e996bf14-e661-4edb-8146-c81fc8c48d2f None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:37.335992 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a7244d-a989-4789-b595-875067a47880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:37.336568 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a7244d-a989-4789-b595-875067a47880 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:37.336849 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a7244d-a989-4789-b595-875067a47880 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:37.337122 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a7244d-a989-4789-b595-875067a47880 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:37.337985 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a7244d-a989-4789-b595-875067a47880 None admin] RBAC: Adding query filter params (role.id=7d6c4b2adb9d43bca1a6573e8f7a9381, user.id=86f547ea49dd496fa6970a314fd870aa, scope.system=all) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:37.338299 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a7244d-a989-4789-b595-875067a47880 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:37.339736 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a7244d-a989-4789-b595-875067a47880 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:37.355702 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 179/347] 10.222.0.27 () {56 vars in 1308 bytes} [Fri Jun 10 12:44:37 2022] GET /identity/v3/role_assignments?user.id=86f547ea49dd496fa6970a314fd870aa&scope.system=all&role.id=7d6c4b2adb9d43bca1a6573e8f7a9381 => generated 510 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:37.355702 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88297f77-ec59-4851-9f6c-bbc2ed4af578 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:37.355702 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88297f77-ec59-4851-9f6c-bbc2ed4af578 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:37.355702 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88297f77-ec59-4851-9f6c-bbc2ed4af578 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:37.355702 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 180/348] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:44:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:44:37.356261 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88297f77-ec59-4851-9f6c-bbc2ed4af578 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:37.437699 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 169/349] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Jun 10 12:44:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 10 12:44:37.439045 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4079d40-67f2-4a67-8ab6-3ae4c15498bd None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:37.439335 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4079d40-67f2-4a67-8ab6-3ae4c15498bd None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:37.439611 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4079d40-67f2-4a67-8ab6-3ae4c15498bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:37.463957 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-b4079d40-67f2-4a67-8ab6-3ae4c15498bd None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:44:37.463957 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b4079d40-67f2-4a67-8ab6-3ae4c15498bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:44:37.552859 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88297f77-ec59-4851-9f6c-bbc2ed4af578 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:37.572724 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:37.585786 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 170/350] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:44:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 10 12:44:37.588352 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4079d40-67f2-4a67-8ab6-3ae4c15498bd None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:37.620745 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1d904a-e07a-4970-a2ad-b80633180352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:37.621415 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1d904a-e07a-4970-a2ad-b80633180352 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:37.621696 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1d904a-e07a-4970-a2ad-b80633180352 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:37.621969 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1d904a-e07a-4970-a2ad-b80633180352 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:37.622879 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1d904a-e07a-4970-a2ad-b80633180352 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:37.623252 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1d904a-e07a-4970-a2ad-b80633180352 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:37.624783 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1d904a-e07a-4970-a2ad-b80633180352 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:37.636172 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:37.642147 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 181/351] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Jun 10 12:44:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:37.643037 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1d904a-e07a-4970-a2ad-b80633180352 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:37.732411 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4079d40-67f2-4a67-8ab6-3ae4c15498bd None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:37.782796 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0cd405-b1de-4d1a-9b69-2ab94f7e322e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:37.783823 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0cd405-b1de-4d1a-9b69-2ab94f7e322e None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:37.784161 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0cd405-b1de-4d1a-9b69-2ab94f7e322e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:37.784437 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0cd405-b1de-4d1a-9b69-2ab94f7e322e None admin] PATH_INFO: `/v3/projects/3663edc029e8431d904a228a9e5755a0` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:37.786316 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0cd405-b1de-4d1a-9b69-2ab94f7e322e None admin] RBAC: Authorizing `identity:get_project(project_id=3663edc029e8431d904a228a9e5755a0)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:37.787775 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1d904a-e07a-4970-a2ad-b80633180352 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:37.788080 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0cd405-b1de-4d1a-9b69-2ab94f7e322e None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:37.790811 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 171/352] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Jun 10 12:44:37 2022] GET /identity/v3/projects/3663edc029e8431d904a228a9e5755a0 => generated 298 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:37.791042 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0cd405-b1de-4d1a-9b69-2ab94f7e322e None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:37.834701 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e350582d-3be2-4b09-854c-3472fccbbac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:37.835779 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e350582d-3be2-4b09-854c-3472fccbbac9 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:37.836260 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e350582d-3be2-4b09-854c-3472fccbbac9 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:37.836792 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e350582d-3be2-4b09-854c-3472fccbbac9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:37.849209 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e350582d-3be2-4b09-854c-3472fccbbac9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:44:37.857339 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-e350582d-3be2-4b09-854c-3472fccbbac9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 10 12:44:37.857339 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:44:37.857339 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:44:37.857339 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:44:37.857339 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:44:37.857339 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:44:37.857339 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:44:37.857339 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:44:37.857339 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:44:37.857339 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:44:37.857339 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:44:37.857339 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:44:37.857339 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:44:37.857339 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:44:37.857339 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:44:37.857339 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:44:37.858263 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:44:37.858263 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:44:37.858263 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:44:37.858263 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:44:37.858263 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:44:37.858263 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:44:37.858263 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:44:37.858263 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:44:37.858263 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:44:37.858263 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:44:37.858263 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:44:37.858263 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:44:37.858263 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:44:37.858263 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:44:37.858263 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:44:37.858263 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:44:37.859089 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:44:37.859089 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:44:37.859089 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:44:37.859089 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:44:37.859089 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:44:37.859089 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:44:37.859089 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:44:37.859089 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:44:37.859089 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:44:37.859089 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:44:37.859089 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 10 12:44:37.859089 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:44:37.859089 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 182/353] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Jun 10 12:44:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:44:37.861888 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1b7590-e0b1-4993-9f24-94e7fcc3cd94 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:37.861888 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1b7590-e0b1-4993-9f24-94e7fcc3cd94 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:37.861888 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1b7590-e0b1-4993-9f24-94e7fcc3cd94 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:37.861888 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 183/354] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:44:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:44:37.862588 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1b7590-e0b1-4993-9f24-94e7fcc3cd94 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:37.879288 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:37.938038 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0cd405-b1de-4d1a-9b69-2ab94f7e322e None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:37.955943 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:37.978649 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1402af-59bc-4b1c-ac05-11f2ee4093c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:37.979623 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1402af-59bc-4b1c-ac05-11f2ee4093c4 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:37.980116 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1402af-59bc-4b1c-ac05-11f2ee4093c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:37.980594 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1402af-59bc-4b1c-ac05-11f2ee4093c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:38.010001 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1b7590-e0b1-4993-9f24-94e7fcc3cd94 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:38.051706 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da77118-72fb-47ae-8f0f-0c46cbe05408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:38.052275 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da77118-72fb-47ae-8f0f-0c46cbe05408 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:38.054524 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da77118-72fb-47ae-8f0f-0c46cbe05408 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:38.054524 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da77118-72fb-47ae-8f0f-0c46cbe05408 None admin] PATH_INFO: `/v3/groups/ab486657b8d9476d9c728d34cfccfa66` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:38.062275 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da77118-72fb-47ae-8f0f-0c46cbe05408 None admin] RBAC: Authorizing `identity:get_group(group_id=ab486657b8d9476d9c728d34cfccfa66)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:38.066313 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da77118-72fb-47ae-8f0f-0c46cbe05408 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:38.066313 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 184/355] 10.222.0.27 () {56 vars in 1180 bytes} [Fri Jun 10 12:44:37 2022] GET /identity/v3/groups/ab486657b8d9476d9c728d34cfccfa66 => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:38.066313 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da77118-72fb-47ae-8f0f-0c46cbe05408 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:38.076495 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1402af-59bc-4b1c-ac05-11f2ee4093c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:38.078566 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1402af-59bc-4b1c-ac05-11f2ee4093c4 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:38.197652 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:38.248536 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 172/356] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Jun 10 12:44:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 10 12:44:38.250182 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b4be1e-da3a-4e4e-a90b-d73d4b9e55be None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:38.250670 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b4be1e-da3a-4e4e-a90b-d73d4b9e55be None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:38.251143 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b4be1e-da3a-4e4e-a90b-d73d4b9e55be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:38.257486 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da77118-72fb-47ae-8f0f-0c46cbe05408 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:38.268087 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-93b4be1e-da3a-4e4e-a90b-d73d4b9e55be None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:44:38.269505 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-93b4be1e-da3a-4e4e-a90b-d73d4b9e55be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:44:38.284914 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:38.295966 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762029b5-161d-4fa4-b498-80e2fab58a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:38.296711 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762029b5-161d-4fa4-b498-80e2fab58a65 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:38.296998 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762029b5-161d-4fa4-b498-80e2fab58a65 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:38.297278 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762029b5-161d-4fa4-b498-80e2fab58a65 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:38.298037 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762029b5-161d-4fa4-b498-80e2fab58a65 None admin] RBAC: Adding query filter params (name=member) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:38.298467 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762029b5-161d-4fa4-b498-80e2fab58a65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:38.299949 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762029b5-161d-4fa4-b498-80e2fab58a65 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:38.310456 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 185/357] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Jun 10 12:44:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:38.311578 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762029b5-161d-4fa4-b498-80e2fab58a65 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:38.358929 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 173/358] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:44:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 10 12:44:38.360114 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b4be1e-da3a-4e4e-a90b-d73d4b9e55be None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:38.450055 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762029b5-161d-4fa4-b498-80e2fab58a65 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:38.458056 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:38.468697 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b4be1e-da3a-4e4e-a90b-d73d4b9e55be None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:38.496365 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa27ea4-a16b-47f1-99a8-194e02ffce15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:38.497030 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa27ea4-a16b-47f1-99a8-194e02ffce15 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:38.497317 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa27ea4-a16b-47f1-99a8-194e02ffce15 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:38.497593 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa27ea4-a16b-47f1-99a8-194e02ffce15 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:38.498563 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa27ea4-a16b-47f1-99a8-194e02ffce15 None admin] RBAC: Adding query filter params (group.id=ab486657b8d9476d9c728d34cfccfa66, role.id=43106c664e1b40de984c0c62701149c8, scope.project.id=3663edc029e8431d904a228a9e5755a0) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:38.498893 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa27ea4-a16b-47f1-99a8-194e02ffce15 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:38.500651 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa27ea4-a16b-47f1-99a8-194e02ffce15 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:38.506828 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 186/359] 10.222.0.27 () {56 vars in 1376 bytes} [Fri Jun 10 12:44:38 2022] GET /identity/v3/role_assignments?group.id=ab486657b8d9476d9c728d34cfccfa66&scope.project.id=3663edc029e8431d904a228a9e5755a0&role.id=43106c664e1b40de984c0c62701149c8 => generated 260 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:38.507634 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa27ea4-a16b-47f1-99a8-194e02ffce15 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:38.508492 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfefb85-2083-4d12-b79d-7334cf24e059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:38.509573 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfefb85-2083-4d12-b79d-7334cf24e059 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:38.509856 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfefb85-2083-4d12-b79d-7334cf24e059 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:38.510130 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfefb85-2083-4d12-b79d-7334cf24e059 None admin] PATH_INFO: `/v3/projects/3663edc029e8431d904a228a9e5755a0` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:38.511364 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfefb85-2083-4d12-b79d-7334cf24e059 None admin] RBAC: Authorizing `identity:get_project(project_id=3663edc029e8431d904a228a9e5755a0)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:38.512864 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfefb85-2083-4d12-b79d-7334cf24e059 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:38.521942 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 174/360] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Jun 10 12:44:38 2022] GET /identity/v3/projects/3663edc029e8431d904a228a9e5755a0 => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:38.521942 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d3992b-e91a-481e-bee6-a92b3c1b2779 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:38.521942 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d3992b-e91a-481e-bee6-a92b3c1b2779 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:38.522270 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d3992b-e91a-481e-bee6-a92b3c1b2779 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:38.525464 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 175/361] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:44:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:44:38.525464 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d3992b-e91a-481e-bee6-a92b3c1b2779 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:38.579239 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:38.648374 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa27ea4-a16b-47f1-99a8-194e02ffce15 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:38.692816 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfea4112-db43-4c9d-980f-69ef23541e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:38.693529 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfea4112-db43-4c9d-980f-69ef23541e81 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:38.693822 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfea4112-db43-4c9d-980f-69ef23541e81 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:38.694108 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfea4112-db43-4c9d-980f-69ef23541e81 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:38.697292 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d3992b-e91a-481e-bee6-a92b3c1b2779 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:38.743117 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9db19b6-ac48-45ac-8217-050820962dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:38.744445 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9db19b6-ac48-45ac-8217-050820962dd6 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:38.744942 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9db19b6-ac48-45ac-8217-050820962dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:38.745417 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9db19b6-ac48-45ac-8217-050820962dd6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:38.759715 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9db19b6-ac48-45ac-8217-050820962dd6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:44:38.769053 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-c9db19b6-ac48-45ac-8217-050820962dd6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 10 12:44:38.769053 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:44:38.769053 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:44:38.769053 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:44:38.769053 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:44:38.769053 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:44:38.769053 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:44:38.769053 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:44:38.769053 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:44:38.769053 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:44:38.769053 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:44:38.769053 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:44:38.769053 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:44:38.769053 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:44:38.769053 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:44:38.769053 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:44:38.769964 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:44:38.769964 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:44:38.769964 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:44:38.769964 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:44:38.769964 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:44:38.769964 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:44:38.769964 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:44:38.769964 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:44:38.769964 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:44:38.769964 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:44:38.769964 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:44:38.769964 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:44:38.769964 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:44:38.769964 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:44:38.769964 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:44:38.769964 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:44:38.770785 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:44:38.770785 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:44:38.770785 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:44:38.770785 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:44:38.770785 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:44:38.770785 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:44:38.770785 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:44:38.770785 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:44:38.770785 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:44:38.770785 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:44:38.770785 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 10 12:44:38.770785 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:44:38.770785 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 176/362] 10.222.0.27 () {56 vars in 1114 bytes} [Fri Jun 10 12:44:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:44:38.771504 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9db19b6-ac48-45ac-8217-050820962dd6 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:38.782672 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfea4112-db43-4c9d-980f-69ef23541e81 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:38.784829 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfea4112-db43-4c9d-980f-69ef23541e81 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:38.801039 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:38.844691 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:38.900407 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 187/363] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Jun 10 12:44:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 10 12:44:38.901707 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac60437e-9eda-47fd-9723-c492972ddd6f None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:38.901994 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac60437e-9eda-47fd-9723-c492972ddd6f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:38.902268 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac60437e-9eda-47fd-9723-c492972ddd6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:38.907534 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9db19b6-ac48-45ac-8217-050820962dd6 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:38.920173 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-ac60437e-9eda-47fd-9723-c492972ddd6f None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:44:38.921354 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-ac60437e-9eda-47fd-9723-c492972ddd6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:44:38.953302 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00454071-8c1c-4f3f-9a36-cf8b6ceb6c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:38.955396 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00454071-8c1c-4f3f-9a36-cf8b6ceb6c0a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:38.956014 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00454071-8c1c-4f3f-9a36-cf8b6ceb6c0a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:38.956436 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00454071-8c1c-4f3f-9a36-cf8b6ceb6c0a None admin] PATH_INFO: `/v3/groups/ab486657b8d9476d9c728d34cfccfa66` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:38.957795 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00454071-8c1c-4f3f-9a36-cf8b6ceb6c0a None admin] RBAC: Authorizing `identity:get_group(group_id=ab486657b8d9476d9c728d34cfccfa66)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:38.959419 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00454071-8c1c-4f3f-9a36-cf8b6ceb6c0a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:38.961054 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 177/364] 10.222.0.27 () {56 vars in 1180 bytes} [Fri Jun 10 12:44:38 2022] GET /identity/v3/groups/ab486657b8d9476d9c728d34cfccfa66 => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:38.961652 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00454071-8c1c-4f3f-9a36-cf8b6ceb6c0a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:39.016808 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 188/365] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:44:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 10 12:44:39.017597 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac60437e-9eda-47fd-9723-c492972ddd6f None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:39.078523 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:39.101067 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00454071-8c1c-4f3f-9a36-cf8b6ceb6c0a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:39.141507 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:39.143278 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc596ea-c7fb-4941-9c58-07667a0663da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:39.144502 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc596ea-c7fb-4941-9c58-07667a0663da None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:39.144791 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc596ea-c7fb-4941-9c58-07667a0663da None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:39.145222 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc596ea-c7fb-4941-9c58-07667a0663da None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:39.146330 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc596ea-c7fb-4941-9c58-07667a0663da None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:39.146649 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc596ea-c7fb-4941-9c58-07667a0663da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:39.148113 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc596ea-c7fb-4941-9c58-07667a0663da None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:39.163962 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 178/366] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Jun 10 12:44:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:39.163962 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc596ea-c7fb-4941-9c58-07667a0663da None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:39.172988 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac60437e-9eda-47fd-9723-c492972ddd6f None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:39.227098 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdca6aa-fdc8-4f9d-b2fe-6e9c4ed0cb17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:39.228414 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdca6aa-fdc8-4f9d-b2fe-6e9c4ed0cb17 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:39.228414 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdca6aa-fdc8-4f9d-b2fe-6e9c4ed0cb17 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:39.228619 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdca6aa-fdc8-4f9d-b2fe-6e9c4ed0cb17 None admin] PATH_INFO: `/v3/projects/485e375bd3394086be6c5cd49bd19e2d` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:39.229773 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdca6aa-fdc8-4f9d-b2fe-6e9c4ed0cb17 None admin] RBAC: Authorizing `identity:get_project(project_id=485e375bd3394086be6c5cd49bd19e2d)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:39.231221 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdca6aa-fdc8-4f9d-b2fe-6e9c4ed0cb17 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:39.233320 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 189/367] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Jun 10 12:44:39 2022] GET /identity/v3/projects/485e375bd3394086be6c5cd49bd19e2d => generated 302 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:39.234479 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdca6aa-fdc8-4f9d-b2fe-6e9c4ed0cb17 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:39.294533 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc596ea-c7fb-4941-9c58-07667a0663da None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:39.343701 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff5278e-ab9c-4b1c-b961-444548db7fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:39.344408 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff5278e-ab9c-4b1c-b961-444548db7fa3 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:39.344698 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff5278e-ab9c-4b1c-b961-444548db7fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:39.344976 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff5278e-ab9c-4b1c-b961-444548db7fa3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:39.346131 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff5278e-ab9c-4b1c-b961-444548db7fa3 None admin] RBAC: Adding query filter params (group.id=ab486657b8d9476d9c728d34cfccfa66, role.id=2d71ee7c2a544fe5ad77d92b24280efe, scope.project.id=3663edc029e8431d904a228a9e5755a0) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:39.346448 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff5278e-ab9c-4b1c-b961-444548db7fa3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:39.347902 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff5278e-ab9c-4b1c-b961-444548db7fa3 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:39.354368 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 179/368] 10.222.0.27 () {56 vars in 1376 bytes} [Fri Jun 10 12:44:39 2022] GET /identity/v3/role_assignments?group.id=ab486657b8d9476d9c728d34cfccfa66&scope.project.id=3663edc029e8431d904a228a9e5755a0&role.id=2d71ee7c2a544fe5ad77d92b24280efe => generated 260 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:39.355174 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff5278e-ab9c-4b1c-b961-444548db7fa3 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:39.365854 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdca6aa-fdc8-4f9d-b2fe-6e9c4ed0cb17 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:39.373948 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:39.423165 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a90acfa-a61d-49eb-9737-2596eb87e959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:39.424645 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a90acfa-a61d-49eb-9737-2596eb87e959 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:39.425147 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a90acfa-a61d-49eb-9737-2596eb87e959 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:39.425909 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a90acfa-a61d-49eb-9737-2596eb87e959 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:39.443855 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a90acfa-a61d-49eb-9737-2596eb87e959 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:44:39.455421 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-9a90acfa-a61d-49eb-9737-2596eb87e959 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 10 12:44:39.455421 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:44:39.455421 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:44:39.455421 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:44:39.455421 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:44:39.455421 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:44:39.455421 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:44:39.455421 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:44:39.455421 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:44:39.455421 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:44:39.455421 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:44:39.455421 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:44:39.455421 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:44:39.455421 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:44:39.455421 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:44:39.455421 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:44:39.456485 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:44:39.456485 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:44:39.456485 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:44:39.456485 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:44:39.456485 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:44:39.456485 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:44:39.456485 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:44:39.456485 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:44:39.456485 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:44:39.456485 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:44:39.456485 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:44:39.456485 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:44:39.456485 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:44:39.456485 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:44:39.456485 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:44:39.456485 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:44:39.457286 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:44:39.457286 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:44:39.457286 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:44:39.457286 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:44:39.457286 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:44:39.457286 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:44:39.457286 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:44:39.457286 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:44:39.457286 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:44:39.457286 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:44:39.457286 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 10 12:44:39.457286 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:44:39.460034 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 190/369] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Jun 10 12:44:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:44:39.461400 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a90acfa-a61d-49eb-9737-2596eb87e959 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:39.476247 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:39.499495 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff5278e-ab9c-4b1c-b961-444548db7fa3 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:39.541686 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e0b115-5986-494f-9427-3eb2b308db28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:39.542402 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e0b115-5986-494f-9427-3eb2b308db28 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:39.542689 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e0b115-5986-494f-9427-3eb2b308db28 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:39.542969 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e0b115-5986-494f-9427-3eb2b308db28 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:39.584732 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a90acfa-a61d-49eb-9737-2596eb87e959 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:39.620126 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e0b115-5986-494f-9427-3eb2b308db28 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:39.621592 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e0b115-5986-494f-9427-3eb2b308db28 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:39.622382 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e969622-a67f-4dc7-bded-462facf78799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:39.623505 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e969622-a67f-4dc7-bded-462facf78799 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:39.623964 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e969622-a67f-4dc7-bded-462facf78799 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:39.624384 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e969622-a67f-4dc7-bded-462facf78799 None admin] PATH_INFO: `/v3/groups/ab486657b8d9476d9c728d34cfccfa66` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:39.625836 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e969622-a67f-4dc7-bded-462facf78799 None admin] RBAC: Authorizing `identity:get_group(group_id=ab486657b8d9476d9c728d34cfccfa66)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:39.627424 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e969622-a67f-4dc7-bded-462facf78799 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:39.629398 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 191/370] 10.222.0.27 () {56 vars in 1180 bytes} [Fri Jun 10 12:44:39 2022] GET /identity/v3/groups/ab486657b8d9476d9c728d34cfccfa66 => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:39.630311 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e969622-a67f-4dc7-bded-462facf78799 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:39.641794 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:39.720305 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:39.743337 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 180/371] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Jun 10 12:44:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 10 12:44:39.744706 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e969622-a67f-4dc7-bded-462facf78799 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:39.748637 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e0b115-5986-494f-9427-3eb2b308db28 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:39.785364 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89d0d46-1316-45c1-a8d7-d0513267b6be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:39.786332 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89d0d46-1316-45c1-a8d7-d0513267b6be None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:39.787499 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89d0d46-1316-45c1-a8d7-d0513267b6be None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:39.787983 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89d0d46-1316-45c1-a8d7-d0513267b6be None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:39.788884 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89d0d46-1316-45c1-a8d7-d0513267b6be None admin] RBAC: Adding query filter params (name=admin) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:39.789352 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89d0d46-1316-45c1-a8d7-d0513267b6be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:39.791412 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89d0d46-1316-45c1-a8d7-d0513267b6be None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:39.801784 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 192/372] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Jun 10 12:44:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:39.802864 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89d0d46-1316-45c1-a8d7-d0513267b6be None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:39.856236 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e0b115-5986-494f-9427-3eb2b308db28 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:39.870501 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:39.906794 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84052ec6-9daa-46fe-a2fb-aaf485ae8d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:39.907465 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84052ec6-9daa-46fe-a2fb-aaf485ae8d4b None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:39.907755 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84052ec6-9daa-46fe-a2fb-aaf485ae8d4b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:39.908055 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84052ec6-9daa-46fe-a2fb-aaf485ae8d4b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:39.908907 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84052ec6-9daa-46fe-a2fb-aaf485ae8d4b None admin] RBAC: Adding query filter params (group.id=ab486657b8d9476d9c728d34cfccfa66, role.id=43106c664e1b40de984c0c62701149c8, scope.project.id=485e375bd3394086be6c5cd49bd19e2d) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:39.909221 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84052ec6-9daa-46fe-a2fb-aaf485ae8d4b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:39.910674 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84052ec6-9daa-46fe-a2fb-aaf485ae8d4b None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:39.919762 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 181/373] 10.222.0.27 () {56 vars in 1376 bytes} [Fri Jun 10 12:44:39 2022] GET /identity/v3/role_assignments?group.id=ab486657b8d9476d9c728d34cfccfa66&scope.project.id=485e375bd3394086be6c5cd49bd19e2d&role.id=43106c664e1b40de984c0c62701149c8 => generated 260 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:39.920854 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84052ec6-9daa-46fe-a2fb-aaf485ae8d4b None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:39.943890 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89d0d46-1316-45c1-a8d7-d0513267b6be None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:39.981793 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb088ba-f516-4771-b599-af0b48eb60ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:39.982670 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb088ba-f516-4771-b599-af0b48eb60ac None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:39.982954 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb088ba-f516-4771-b599-af0b48eb60ac None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:39.983230 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb088ba-f516-4771-b599-af0b48eb60ac None admin] PATH_INFO: `/v3/projects/485e375bd3394086be6c5cd49bd19e2d` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:39.984481 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb088ba-f516-4771-b599-af0b48eb60ac None admin] RBAC: Authorizing `identity:get_project(project_id=485e375bd3394086be6c5cd49bd19e2d)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:39.991890 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb088ba-f516-4771-b599-af0b48eb60ac None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:39.991890 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 193/374] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Jun 10 12:44:39 2022] GET /identity/v3/projects/485e375bd3394086be6c5cd49bd19e2d => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:39.993465 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb088ba-f516-4771-b599-af0b48eb60ac None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:40.011368 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:40.082609 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84052ec6-9daa-46fe-a2fb-aaf485ae8d4b None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:40.103711 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb088ba-f516-4771-b599-af0b48eb60ac None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:40.143127 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dff7d6-5832-4a0f-8afe-47cee5b8a8b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:40.144049 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dff7d6-5832-4a0f-8afe-47cee5b8a8b3 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:40.144336 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dff7d6-5832-4a0f-8afe-47cee5b8a8b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:40.144616 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dff7d6-5832-4a0f-8afe-47cee5b8a8b3 None admin] PATH_INFO: `/v3/groups/ab486657b8d9476d9c728d34cfccfa66` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:40.146442 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dff7d6-5832-4a0f-8afe-47cee5b8a8b3 None admin] RBAC: Authorizing `identity:get_group(group_id=ab486657b8d9476d9c728d34cfccfa66)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:40.147938 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dff7d6-5832-4a0f-8afe-47cee5b8a8b3 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:40.150450 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 194/375] 10.222.0.27 () {56 vars in 1180 bytes} [Fri Jun 10 12:44:39 2022] GET /identity/v3/groups/ab486657b8d9476d9c728d34cfccfa66 => generated 226 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:40.156187 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dff7d6-5832-4a0f-8afe-47cee5b8a8b3 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:40.157541 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b781c1c0-9143-4b42-af4d-e9e82d8a4c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:40.158379 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b781c1c0-9143-4b42-af4d-e9e82d8a4c11 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:40.158834 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b781c1c0-9143-4b42-af4d-e9e82d8a4c11 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:40.159259 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b781c1c0-9143-4b42-af4d-e9e82d8a4c11 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:40.262560 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:40.300341 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b781c1c0-9143-4b42-af4d-e9e82d8a4c11 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:40.302039 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b781c1c0-9143-4b42-af4d-e9e82d8a4c11 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:40.303630 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:40.366563 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dff7d6-5832-4a0f-8afe-47cee5b8a8b3 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:40.441165 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76a609a-cedd-41b8-b721-24957403dc04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:40.442053 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76a609a-cedd-41b8-b721-24957403dc04 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:40.442478 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76a609a-cedd-41b8-b721-24957403dc04 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:40.442896 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76a609a-cedd-41b8-b721-24957403dc04 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:40.443910 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76a609a-cedd-41b8-b721-24957403dc04 None admin] RBAC: Adding query filter params (group.id=ab486657b8d9476d9c728d34cfccfa66, role.id=2d71ee7c2a544fe5ad77d92b24280efe, scope.project.id=485e375bd3394086be6c5cd49bd19e2d) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:40.444365 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76a609a-cedd-41b8-b721-24957403dc04 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:40.448028 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76a609a-cedd-41b8-b721-24957403dc04 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:40.461319 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 195/376] 10.222.0.27 () {56 vars in 1376 bytes} [Fri Jun 10 12:44:40 2022] GET /identity/v3/role_assignments?group.id=ab486657b8d9476d9c728d34cfccfa66&scope.project.id=485e375bd3394086be6c5cd49bd19e2d&role.id=2d71ee7c2a544fe5ad77d92b24280efe => generated 260 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 12:44:40.480776 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 182/377] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Jun 10 12:44:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 560 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 10 12:44:40.490551 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76a609a-cedd-41b8-b721-24957403dc04 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:40.601725 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76a609a-cedd-41b8-b721-24957403dc04 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:40.646302 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171e462d-aee6-4dd2-afd1-e4fd93535b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:40.647423 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171e462d-aee6-4dd2-afd1-e4fd93535b12 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:40.647892 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171e462d-aee6-4dd2-afd1-e4fd93535b12 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:40.648310 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171e462d-aee6-4dd2-afd1-e4fd93535b12 None admin] PATH_INFO: `/v3/projects/2e3a9af772214953842e9cc63a52ae3a` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:40.649669 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171e462d-aee6-4dd2-afd1-e4fd93535b12 None admin] RBAC: Authorizing `identity:get_project(project_id=2e3a9af772214953842e9cc63a52ae3a)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:40.651805 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171e462d-aee6-4dd2-afd1-e4fd93535b12 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:40.653973 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 196/378] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Jun 10 12:44:40 2022] GET /identity/v3/projects/2e3a9af772214953842e9cc63a52ae3a => generated 344 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:40.659546 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b781c1c0-9143-4b42-af4d-e9e82d8a4c11 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:40.761218 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:40.785733 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b781c1c0-9143-4b42-af4d-e9e82d8a4c11 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:40.809516 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b41e9e-9619-4b1d-82f2-5284bd8f44b5 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:40.809808 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b41e9e-9619-4b1d-82f2-5284bd8f44b5 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:40.810088 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b41e9e-9619-4b1d-82f2-5284bd8f44b5 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:40.811691 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 197/379] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:44:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 10 12:44:40.817428 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c075b05f-5fb1-4cb5-831c-c464fe9da54d None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:40.817969 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c075b05f-5fb1-4cb5-831c-c464fe9da54d None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:40.818409 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c075b05f-5fb1-4cb5-831c-c464fe9da54d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:40.828990 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21f13e3-5404-4a26-ad21-fe7a6d1cf216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:40.829917 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21f13e3-5404-4a26-ad21-fe7a6d1cf216 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:40.830203 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21f13e3-5404-4a26-ad21-fe7a6d1cf216 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:40.830480 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21f13e3-5404-4a26-ad21-fe7a6d1cf216 None admin] PATH_INFO: `/v3/groups/52dd74368faf4035b19b7e1f005cf049` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:40.834107 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:40.838469 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21f13e3-5404-4a26-ad21-fe7a6d1cf216 None admin] RBAC: Authorizing `identity:get_group(group_id=52dd74368faf4035b19b7e1f005cf049)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:40.840091 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-c075b05f-5fb1-4cb5-831c-c464fe9da54d None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:44:40.840225 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21f13e3-5404-4a26-ad21-fe7a6d1cf216 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:40.841636 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c075b05f-5fb1-4cb5-831c-c464fe9da54d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:44:40.841991 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 183/380] 10.222.0.27 () {56 vars in 1180 bytes} [Fri Jun 10 12:44:40 2022] GET /identity/v3/groups/52dd74368faf4035b19b7e1f005cf049 => generated 229 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:40.848956 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21f13e3-5404-4a26-ad21-fe7a6d1cf216 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:40.958688 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21f13e3-5404-4a26-ad21-fe7a6d1cf216 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:40.979669 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 198/381] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:44:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 10 12:44:40.995050 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20a5537-a03d-4272-9e65-ea8f6e57fae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:40.995714 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20a5537-a03d-4272-9e65-ea8f6e57fae2 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:40.996084 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20a5537-a03d-4272-9e65-ea8f6e57fae2 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:40.996365 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20a5537-a03d-4272-9e65-ea8f6e57fae2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:40.997419 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20a5537-a03d-4272-9e65-ea8f6e57fae2 None admin] RBAC: Adding query filter params (group.id=52dd74368faf4035b19b7e1f005cf049, role.id=b43d5a8247e244869c2c70f986f42e8a, scope.project.id=2e3a9af772214953842e9cc63a52ae3a) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:40.997877 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20a5537-a03d-4272-9e65-ea8f6e57fae2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:40.999465 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20a5537-a03d-4272-9e65-ea8f6e57fae2 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:41.005724 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 184/382] 10.222.0.27 () {56 vars in 1376 bytes} [Fri Jun 10 12:44:40 2022] GET /identity/v3/role_assignments?group.id=52dd74368faf4035b19b7e1f005cf049&scope.project.id=2e3a9af772214953842e9cc63a52ae3a&role.id=b43d5a8247e244869c2c70f986f42e8a => generated 260 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 12:44:41.016217 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c075b05f-5fb1-4cb5-831c-c464fe9da54d None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:41.158030 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c075b05f-5fb1-4cb5-831c-c464fe9da54d None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:41.195117 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b239a794-fcdb-4bd6-98bd-1f0b2df02ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:41.195994 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b239a794-fcdb-4bd6-98bd-1f0b2df02ae6 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:41.196227 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b239a794-fcdb-4bd6-98bd-1f0b2df02ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:41.196592 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b239a794-fcdb-4bd6-98bd-1f0b2df02ae6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:41.207381 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b239a794-fcdb-4bd6-98bd-1f0b2df02ae6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:44:41.213298 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:41.216617 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-b239a794-fcdb-4bd6-98bd-1f0b2df02ae6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 10 12:44:41.216617 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:44:41.216617 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:44:41.216617 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:44:41.216617 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:44:41.216617 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:44:41.216617 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:44:41.216617 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:44:41.216617 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:44:41.216617 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:44:41.216617 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:44:41.216617 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:44:41.216617 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:44:41.216617 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:44:41.216617 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:44:41.216617 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:44:41.217683 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:44:41.217683 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:44:41.217683 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:44:41.217683 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:44:41.217683 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:44:41.217683 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:44:41.217683 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:44:41.217683 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:44:41.217683 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:44:41.217683 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:44:41.217683 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:44:41.217683 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:44:41.217683 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:44:41.217683 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:44:41.217683 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:44:41.217683 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:44:41.218702 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:44:41.218702 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:44:41.218702 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:44:41.218702 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:44:41.218702 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:44:41.218702 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:44:41.218702 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:44:41.218702 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:44:41.218702 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:44:41.218702 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:44:41.218702 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 10 12:44:41.218702 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:44:41.221222 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 199/383] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Jun 10 12:44:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:44:41.227226 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20a5537-a03d-4272-9e65-ea8f6e57fae2 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:41.232203 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:41.341302 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20a5537-a03d-4272-9e65-ea8f6e57fae2 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:41.390439 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be522bd6-3101-4bd9-8d3e-393ec3103d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:41.391124 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be522bd6-3101-4bd9-8d3e-393ec3103d83 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:41.391407 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be522bd6-3101-4bd9-8d3e-393ec3103d83 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:41.391680 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be522bd6-3101-4bd9-8d3e-393ec3103d83 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:41.392460 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be522bd6-3101-4bd9-8d3e-393ec3103d83 None admin] RBAC: Adding query filter params (name=member) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:41.392762 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be522bd6-3101-4bd9-8d3e-393ec3103d83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:41.394205 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be522bd6-3101-4bd9-8d3e-393ec3103d83 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:41.407263 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 185/384] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Jun 10 12:44:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:41.412806 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b239a794-fcdb-4bd6-98bd-1f0b2df02ae6 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:41.549408 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b239a794-fcdb-4bd6-98bd-1f0b2df02ae6 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:41.593615 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197dcfb9-c9f7-4c25-b205-d243bda044fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:41.594460 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197dcfb9-c9f7-4c25-b205-d243bda044fe None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:41.594747 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197dcfb9-c9f7-4c25-b205-d243bda044fe None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:41.595025 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197dcfb9-c9f7-4c25-b205-d243bda044fe None admin] PATH_INFO: `/v3/groups/ab486657b8d9476d9c728d34cfccfa66` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:41.597105 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197dcfb9-c9f7-4c25-b205-d243bda044fe None admin] RBAC: Authorizing `identity:get_group(group_id=ab486657b8d9476d9c728d34cfccfa66)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:41.598566 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197dcfb9-c9f7-4c25-b205-d243bda044fe None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:41.602295 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 200/385] 10.222.0.27 () {56 vars in 1180 bytes} [Fri Jun 10 12:44:41 2022] GET /identity/v3/groups/ab486657b8d9476d9c728d34cfccfa66 => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:41.607699 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be522bd6-3101-4bd9-8d3e-393ec3103d83 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:41.712338 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:41.748646 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be522bd6-3101-4bd9-8d3e-393ec3103d83 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:41.799402 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a7e884-7850-4fab-a352-7dc1c1486e50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:41.800297 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a7e884-7850-4fab-a352-7dc1c1486e50 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:41.800578 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a7e884-7850-4fab-a352-7dc1c1486e50 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:41.800852 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a7e884-7850-4fab-a352-7dc1c1486e50 None admin] PATH_INFO: `/v3/projects/3663edc029e8431d904a228a9e5755a0` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:41.802042 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a7e884-7850-4fab-a352-7dc1c1486e50 None admin] RBAC: Authorizing `identity:get_project(project_id=3663edc029e8431d904a228a9e5755a0)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:41.803562 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a7e884-7850-4fab-a352-7dc1c1486e50 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:41.805389 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 186/386] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Jun 10 12:44:41 2022] GET /identity/v3/projects/3663edc029e8431d904a228a9e5755a0 => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:41.811502 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197dcfb9-c9f7-4c25-b205-d243bda044fe None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:41.817190 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb362b30-689f-4b37-92e0-64589ece5f62 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:41.817481 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb362b30-689f-4b37-92e0-64589ece5f62 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:41.817825 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb362b30-689f-4b37-92e0-64589ece5f62 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:41.820564 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 187/387] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:44:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:44:41.831306 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a515cd74-43d0-4feb-9c30-629efd20e0e8 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:41.831599 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a515cd74-43d0-4feb-9c30-629efd20e0e8 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:41.832279 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a515cd74-43d0-4feb-9c30-629efd20e0e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:41.865263 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-a515cd74-43d0-4feb-9c30-629efd20e0e8 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:44:41.866479 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a515cd74-43d0-4feb-9c30-629efd20e0e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:44:41.936212 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:42.004259 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 188/388] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:44:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 10 12:44:42.006169 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197dcfb9-c9f7-4c25-b205-d243bda044fe None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:42.050240 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a515cd74-43d0-4feb-9c30-629efd20e0e8 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:42.050438 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7356aba3-c51d-42ae-9bb7-8f53d892ad77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:42.050438 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7356aba3-c51d-42ae-9bb7-8f53d892ad77 None admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:42.050438 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7356aba3-c51d-42ae-9bb7-8f53d892ad77 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:42.050438 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7356aba3-c51d-42ae-9bb7-8f53d892ad77 None admin] PATH_INFO: `/v3/projects/3663edc029e8431d904a228a9e5755a0/groups/ab486657b8d9476d9c728d34cfccfa66/roles/43106c664e1b40de984c0c62701149c8` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:42.052005 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7356aba3-c51d-42ae-9bb7-8f53d892ad77 None admin] RBAC: Authorizing `identity:create_grant(project_id=3663edc029e8431d904a228a9e5755a0, group_id=ab486657b8d9476d9c728d34cfccfa66, role_id=43106c664e1b40de984c0c62701149c8)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:42.066345 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7356aba3-c51d-42ae-9bb7-8f53d892ad77 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:42.123590 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:42.139686 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 201/389] 10.222.0.27 () {58 vars in 1442 bytes} [Fri Jun 10 12:44:41 2022] PUT /identity/v3/projects/3663edc029e8431d904a228a9e5755a0/groups/ab486657b8d9476d9c728d34cfccfa66/roles/43106c664e1b40de984c0c62701149c8 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 12:44:42.172760 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a515cd74-43d0-4feb-9c30-629efd20e0e8 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:42.212661 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d02bf92-b2d7-45a8-bd38-d78a892336e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:42.213521 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d02bf92-b2d7-45a8-bd38-d78a892336e0 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:42.213805 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d02bf92-b2d7-45a8-bd38-d78a892336e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:42.214083 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d02bf92-b2d7-45a8-bd38-d78a892336e0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:42.225452 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d02bf92-b2d7-45a8-bd38-d78a892336e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:44:42.235081 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-4d02bf92-b2d7-45a8-bd38-d78a892336e0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 10 12:44:42.235081 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:44:42.235081 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:44:42.235081 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:44:42.235081 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:44:42.235081 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:44:42.235081 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:44:42.235081 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:44:42.235081 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:44:42.235081 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:44:42.235081 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:44:42.235081 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:44:42.235081 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:44:42.235081 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:44:42.235081 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:44:42.235081 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:44:42.236589 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:44:42.236589 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:44:42.236589 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:44:42.236589 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:44:42.236589 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:44:42.236589 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:44:42.236589 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:44:42.236589 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:44:42.236589 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:44:42.236589 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:44:42.236589 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:44:42.236589 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:44:42.236589 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:44:42.236589 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:44:42.236589 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:44:42.236589 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:44:42.237428 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:44:42.237428 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:44:42.237428 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:44:42.237428 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:44:42.237428 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:44:42.237428 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:44:42.237428 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:44:42.237428 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:44:42.237428 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:44:42.237428 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:44:42.237428 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 10 12:44:42.237428 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:44:42.237428 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 189/390] 10.222.0.27 () {56 vars in 1114 bytes} [Fri Jun 10 12:44:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:44:42.245496 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7356aba3-c51d-42ae-9bb7-8f53d892ad77 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:42.380908 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7356aba3-c51d-42ae-9bb7-8f53d892ad77 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:42.411789 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:42.423899 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6247874a-ef8d-45c8-a742-def857a9927c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:42.424635 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6247874a-ef8d-45c8-a742-def857a9927c None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:42.424921 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6247874a-ef8d-45c8-a742-def857a9927c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:42.425204 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6247874a-ef8d-45c8-a742-def857a9927c None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:42.425964 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6247874a-ef8d-45c8-a742-def857a9927c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:42.426273 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6247874a-ef8d-45c8-a742-def857a9927c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:42.427647 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6247874a-ef8d-45c8-a742-def857a9927c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:42.437366 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 202/391] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Jun 10 12:44:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:42.442896 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d02bf92-b2d7-45a8-bd38-d78a892336e0 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:42.568781 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d02bf92-b2d7-45a8-bd38-d78a892336e0 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:42.602858 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:42.619013 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4d9173-b345-42d0-91d1-8266868490dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:42.619898 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4d9173-b345-42d0-91d1-8266868490dd None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:42.620194 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4d9173-b345-42d0-91d1-8266868490dd None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:42.620474 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4d9173-b345-42d0-91d1-8266868490dd None admin] PATH_INFO: `/v3/groups/ab486657b8d9476d9c728d34cfccfa66` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:42.621687 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4d9173-b345-42d0-91d1-8266868490dd None admin] RBAC: Authorizing `identity:get_group(group_id=ab486657b8d9476d9c728d34cfccfa66)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:42.623125 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4d9173-b345-42d0-91d1-8266868490dd None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:42.625082 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 190/392] 10.222.0.27 () {56 vars in 1180 bytes} [Fri Jun 10 12:44:42 2022] GET /identity/v3/groups/ab486657b8d9476d9c728d34cfccfa66 => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 12:44:42.633243 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6247874a-ef8d-45c8-a742-def857a9927c None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:42.785495 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6247874a-ef8d-45c8-a742-def857a9927c None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:42.829452 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d5e457-0eca-4f04-8d70-3587cd66b83c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:42.830336 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d5e457-0eca-4f04-8d70-3587cd66b83c None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:42.830617 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d5e457-0eca-4f04-8d70-3587cd66b83c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:42.830892 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d5e457-0eca-4f04-8d70-3587cd66b83c None admin] PATH_INFO: `/v3/projects/3663edc029e8431d904a228a9e5755a0` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:42.832100 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d5e457-0eca-4f04-8d70-3587cd66b83c None admin] RBAC: Authorizing `identity:get_project(project_id=3663edc029e8431d904a228a9e5755a0)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:42.833524 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d5e457-0eca-4f04-8d70-3587cd66b83c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:42.835514 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 203/393] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Jun 10 12:44:42 2022] GET /identity/v3/projects/3663edc029e8431d904a228a9e5755a0 => generated 298 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:42.841733 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4d9173-b345-42d0-91d1-8266868490dd None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:42.998251 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4d9173-b345-42d0-91d1-8266868490dd None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:43.039769 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0170deaf-f902-486b-a845-82474b4e71fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:43.040707 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0170deaf-f902-486b-a845-82474b4e71fd None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:43.040992 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0170deaf-f902-486b-a845-82474b4e71fd None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:43.041267 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0170deaf-f902-486b-a845-82474b4e71fd None admin] PATH_INFO: `/v3/projects/3663edc029e8431d904a228a9e5755a0/groups/ab486657b8d9476d9c728d34cfccfa66/roles/2d71ee7c2a544fe5ad77d92b24280efe` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:43.043497 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0170deaf-f902-486b-a845-82474b4e71fd None admin] RBAC: Authorizing `identity:create_grant(project_id=3663edc029e8431d904a228a9e5755a0, group_id=ab486657b8d9476d9c728d34cfccfa66, role_id=2d71ee7c2a544fe5ad77d92b24280efe)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:43.044961 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0170deaf-f902-486b-a845-82474b4e71fd None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:43.128760 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 191/394] 10.222.0.27 () {58 vars in 1442 bytes} [Fri Jun 10 12:44:42 2022] PUT /identity/v3/projects/3663edc029e8431d904a228a9e5755a0/groups/ab486657b8d9476d9c728d34cfccfa66/roles/2d71ee7c2a544fe5ad77d92b24280efe => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 12:44:43.299698 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ba715e-3d51-44c8-989b-c156f3230666 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:43.300206 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ba715e-3d51-44c8-989b-c156f3230666 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:43.300206 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ba715e-3d51-44c8-989b-c156f3230666 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:43.301805 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 204/395] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:44:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:44:43.307891 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348acb41-3e9f-4058-98c8-a03b96dfd417 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:43.308307 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348acb41-3e9f-4058-98c8-a03b96dfd417 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:43.308537 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348acb41-3e9f-4058-98c8-a03b96dfd417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:43.328823 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9ff974-b6f4-4d72-9cc8-071bf8ef1197 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:43.329438 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9ff974-b6f4-4d72-9cc8-071bf8ef1197 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:43.329917 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9ff974-b6f4-4d72-9cc8-071bf8ef1197 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:43.331525 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-348acb41-3e9f-4058-98c8-a03b96dfd417 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:44:43.332108 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 205/396] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:44:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:44:43.334696 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-348acb41-3e9f-4058-98c8-a03b96dfd417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:44:43.337067 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f55c19-3a48-459a-b3ef-dd45bcf5271a None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:43.337611 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f55c19-3a48-459a-b3ef-dd45bcf5271a None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:43.338089 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f55c19-3a48-459a-b3ef-dd45bcf5271a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:43.370379 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-41f55c19-3a48-459a-b3ef-dd45bcf5271a None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:44:43.372036 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-41f55c19-3a48-459a-b3ef-dd45bcf5271a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:44:43.404262 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:43.414560 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 192/397] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:44:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 10 12:44:43.438747 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348acb41-3e9f-4058-98c8-a03b96dfd417 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:43.468987 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:43.495068 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 206/398] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:44:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 10 12:44:43.516678 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f55c19-3a48-459a-b3ef-dd45bcf5271a None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:43.558284 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348acb41-3e9f-4058-98c8-a03b96dfd417 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:43.612543 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273597e4-83f1-4b46-819f-f9691dd0e5f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:43.613345 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273597e4-83f1-4b46-819f-f9691dd0e5f9 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:43.613623 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273597e4-83f1-4b46-819f-f9691dd0e5f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:43.613904 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273597e4-83f1-4b46-819f-f9691dd0e5f9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:43.626907 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273597e4-83f1-4b46-819f-f9691dd0e5f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:44:43.635166 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-273597e4-83f1-4b46-819f-f9691dd0e5f9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 10 12:44:43.635166 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:44:43.635166 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:44:43.635166 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:44:43.635166 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:44:43.635166 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:44:43.635166 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:44:43.635166 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:44:43.635166 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:44:43.635166 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:44:43.635166 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:44:43.635166 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:44:43.635166 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:44:43.635166 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:44:43.635166 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:44:43.635166 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:44:43.636120 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:44:43.636120 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:44:43.636120 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:44:43.636120 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:44:43.636120 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:44:43.636120 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:44:43.636120 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:44:43.636120 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:44:43.636120 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:44:43.636120 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:44:43.636120 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:44:43.636120 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:44:43.636120 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:44:43.636120 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:44:43.636120 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:44:43.636120 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:44:43.636934 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:44:43.636934 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:44:43.636934 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:44:43.636934 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:44:43.636934 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:44:43.636934 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:44:43.636934 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:44:43.636934 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:44:43.636934 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:44:43.636934 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:44:43.636934 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 10 12:44:43.636934 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:44:43.636934 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 193/399] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Jun 10 12:44:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 10 12:44:43.638143 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdf1bf1-baa9-4283-a05e-f26b77e19864 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:43.638143 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdf1bf1-baa9-4283-a05e-f26b77e19864 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:43.638296 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdf1bf1-baa9-4283-a05e-f26b77e19864 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:43.639883 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 194/400] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:44:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:44:43.641943 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdf1bf1-baa9-4283-a05e-f26b77e19864 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:43.651091 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f55c19-3a48-459a-b3ef-dd45bcf5271a None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:43.687604 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32340f2c-4afc-4061-a818-b2fc2b5d9556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:43.688869 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32340f2c-4afc-4061-a818-b2fc2b5d9556 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:43.689358 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32340f2c-4afc-4061-a818-b2fc2b5d9556 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:43.689913 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32340f2c-4afc-4061-a818-b2fc2b5d9556 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:43.692785 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:43.701386 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32340f2c-4afc-4061-a818-b2fc2b5d9556 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:44:43.710497 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-32340f2c-4afc-4061-a818-b2fc2b5d9556 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 10 12:44:43.710497 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:44:43.710497 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:44:43.710497 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:44:43.710497 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:44:43.710497 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:44:43.710497 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:44:43.710497 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:44:43.710497 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:44:43.710497 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:44:43.710497 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:44:43.710497 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:44:43.710497 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:44:43.710497 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:44:43.710497 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:44:43.710497 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:44:43.711423 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:44:43.711423 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:44:43.711423 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:44:43.711423 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:44:43.711423 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:44:43.711423 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:44:43.711423 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:44:43.711423 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:44:43.711423 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:44:43.711423 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:44:43.711423 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:44:43.711423 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:44:43.711423 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:44:43.711423 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:44:43.711423 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:44:43.711423 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:44:43.712291 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:44:43.712291 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:44:43.712291 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:44:43.712291 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:44:43.712291 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:44:43.712291 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:44:43.712291 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:44:43.712291 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:44:43.712291 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:44:43.712291 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:44:43.712291 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 10 12:44:43.712291 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:44:43.715090 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 207/401] 10.222.0.27 () {56 vars in 1114 bytes} [Fri Jun 10 12:44:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 10 12:44:43.716666 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739e4c69-63ce-4fcb-b936-b19ac1ddbf59 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:43.717177 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739e4c69-63ce-4fcb-b936-b19ac1ddbf59 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:43.717656 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739e4c69-63ce-4fcb-b936-b19ac1ddbf59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:43.736628 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-739e4c69-63ce-4fcb-b936-b19ac1ddbf59 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:44:43.738173 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-739e4c69-63ce-4fcb-b936-b19ac1ddbf59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:44:43.762899 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdf1bf1-baa9-4283-a05e-f26b77e19864 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:43.804907 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f5a4db-c6e0-45c5-b63d-e483ab53f2fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:43.805714 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f5a4db-c6e0-45c5-b63d-e483ab53f2fa None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:43.806676 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f5a4db-c6e0-45c5-b63d-e483ab53f2fa None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:43.806963 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f5a4db-c6e0-45c5-b63d-e483ab53f2fa None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:43.807708 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f5a4db-c6e0-45c5-b63d-e483ab53f2fa None admin] RBAC: Adding query filter params (name=member) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:43.808045 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f5a4db-c6e0-45c5-b63d-e483ab53f2fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:43.809522 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f5a4db-c6e0-45c5-b63d-e483ab53f2fa None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:43.811788 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 208/402] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:44:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 10 12:44:43.812139 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:43.822334 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 195/403] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Jun 10 12:44:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:43.822487 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739e4c69-63ce-4fcb-b936-b19ac1ddbf59 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:43.826407 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f5a4db-c6e0-45c5-b63d-e483ab53f2fa None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:43.932318 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739e4c69-63ce-4fcb-b936-b19ac1ddbf59 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:43.936444 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f5a4db-c6e0-45c5-b63d-e483ab53f2fa None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:43.940181 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:43.967015 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c06736d-f7ab-4d6f-99ba-dcebedc3642b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:43.967687 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c06736d-f7ab-4d6f-99ba-dcebedc3642b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:43.968016 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c06736d-f7ab-4d6f-99ba-dcebedc3642b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:43.968293 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c06736d-f7ab-4d6f-99ba-dcebedc3642b None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:43.969036 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c06736d-f7ab-4d6f-99ba-dcebedc3642b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:43.969340 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c06736d-f7ab-4d6f-99ba-dcebedc3642b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:43.970827 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c06736d-f7ab-4d6f-99ba-dcebedc3642b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:43.972781 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e21258f-755b-4365-949f-4ca45de2d359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:43.975916 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e21258f-755b-4365-949f-4ca45de2d359 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:43.975916 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e21258f-755b-4365-949f-4ca45de2d359 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:43.975916 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e21258f-755b-4365-949f-4ca45de2d359 None admin] PATH_INFO: `/v3/groups/ab486657b8d9476d9c728d34cfccfa66` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:43.975916 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e21258f-755b-4365-949f-4ca45de2d359 None admin] RBAC: Authorizing `identity:get_group(group_id=ab486657b8d9476d9c728d34cfccfa66)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:43.977076 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e21258f-755b-4365-949f-4ca45de2d359 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:43.979575 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 209/404] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Jun 10 12:44:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:43.979963 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 196/405] 10.222.0.27 () {56 vars in 1180 bytes} [Fri Jun 10 12:44:43 2022] GET /identity/v3/groups/ab486657b8d9476d9c728d34cfccfa66 => generated 226 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:43.984486 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c06736d-f7ab-4d6f-99ba-dcebedc3642b None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:43.985624 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e21258f-755b-4365-949f-4ca45de2d359 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:44.021546 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:44.090080 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c06736d-f7ab-4d6f-99ba-dcebedc3642b None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:44.100435 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e21258f-755b-4365-949f-4ca45de2d359 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:44.128733 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49874373-6150-40f4-9663-daefee55c921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:44.129734 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49874373-6150-40f4-9663-daefee55c921 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:44.130087 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49874373-6150-40f4-9663-daefee55c921 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:44.130430 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49874373-6150-40f4-9663-daefee55c921 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:44.138879 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f871ec5b-7292-4610-a3d5-3db13b6c738b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:44.140124 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f871ec5b-7292-4610-a3d5-3db13b6c738b None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:44.140598 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f871ec5b-7292-4610-a3d5-3db13b6c738b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:44.141070 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f871ec5b-7292-4610-a3d5-3db13b6c738b None admin] PATH_INFO: `/v3/projects/485e375bd3394086be6c5cd49bd19e2d` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:44.142452 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f871ec5b-7292-4610-a3d5-3db13b6c738b None admin] RBAC: Authorizing `identity:get_project(project_id=485e375bd3394086be6c5cd49bd19e2d)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:44.142574 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49874373-6150-40f4-9663-daefee55c921 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:44:44.144235 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f871ec5b-7292-4610-a3d5-3db13b6c738b None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:44.146622 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 197/406] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Jun 10 12:44:43 2022] GET /identity/v3/projects/485e375bd3394086be6c5cd49bd19e2d => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:44.147633 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f871ec5b-7292-4610-a3d5-3db13b6c738b None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:44.152518 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-49874373-6150-40f4-9663-daefee55c921 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 10 12:44:44.152518 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:44:44.152518 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:44:44.152518 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:44:44.152518 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:44:44.152518 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:44:44.152518 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:44:44.152518 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:44:44.152518 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:44:44.152518 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:44:44.152518 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:44:44.152518 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:44:44.152518 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:44:44.152518 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:44:44.152518 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:44:44.152518 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:44:44.153484 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:44:44.153484 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:44:44.153484 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:44:44.153484 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:44:44.153484 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:44:44.153484 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:44:44.153484 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:44:44.153484 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:44:44.153484 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:44:44.153484 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:44:44.153484 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:44:44.153484 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:44:44.153484 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:44:44.153484 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:44:44.153484 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:44:44.153484 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:44:44.154343 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:44:44.154343 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:44:44.154343 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:44:44.154343 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:44:44.154343 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:44:44.154343 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:44:44.154343 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:44:44.154343 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:44:44.154343 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:44:44.154343 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:44:44.154343 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 10 12:44:44.154343 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:44:44.154343 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 210/407] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Jun 10 12:44:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:44:44.154343 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49874373-6150-40f4-9663-daefee55c921 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:44.201708 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:44.256698 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:44.263531 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49874373-6150-40f4-9663-daefee55c921 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:44.268367 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f871ec5b-7292-4610-a3d5-3db13b6c738b None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:44.300607 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca247f0-ab60-43c5-bd7c-65abf7e29601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:44.301732 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca247f0-ab60-43c5-bd7c-65abf7e29601 None admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:44.302215 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca247f0-ab60-43c5-bd7c-65abf7e29601 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:44.302589 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca247f0-ab60-43c5-bd7c-65abf7e29601 None admin] PATH_INFO: `/v3/projects/485e375bd3394086be6c5cd49bd19e2d/groups/ab486657b8d9476d9c728d34cfccfa66/roles/43106c664e1b40de984c0c62701149c8` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:44.303059 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21484808-f780-4be5-883f-c551ac3fc79b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:44.304101 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21484808-f780-4be5-883f-c551ac3fc79b None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:44.304525 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21484808-f780-4be5-883f-c551ac3fc79b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:44.304959 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca247f0-ab60-43c5-bd7c-65abf7e29601 None admin] RBAC: Authorizing `identity:create_grant(project_id=485e375bd3394086be6c5cd49bd19e2d, group_id=ab486657b8d9476d9c728d34cfccfa66, role_id=43106c664e1b40de984c0c62701149c8)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:44.305082 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21484808-f780-4be5-883f-c551ac3fc79b None admin] PATH_INFO: `/v3/groups/ab486657b8d9476d9c728d34cfccfa66` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:44.306489 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21484808-f780-4be5-883f-c551ac3fc79b None admin] RBAC: Authorizing `identity:get_group(group_id=ab486657b8d9476d9c728d34cfccfa66)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:44.306787 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca247f0-ab60-43c5-bd7c-65abf7e29601 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:44.308398 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21484808-f780-4be5-883f-c551ac3fc79b None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:44.310817 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 198/408] 10.222.0.27 () {56 vars in 1180 bytes} [Fri Jun 10 12:44:44 2022] GET /identity/v3/groups/ab486657b8d9476d9c728d34cfccfa66 => generated 226 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:44.311779 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21484808-f780-4be5-883f-c551ac3fc79b None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:44.400735 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 211/409] 10.222.0.27 () {58 vars in 1442 bytes} [Fri Jun 10 12:44:44 2022] PUT /identity/v3/projects/485e375bd3394086be6c5cd49bd19e2d/groups/ab486657b8d9476d9c728d34cfccfa66/roles/43106c664e1b40de984c0c62701149c8 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 12:44:44.401858 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca247f0-ab60-43c5-bd7c-65abf7e29601 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:44.420528 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21484808-f780-4be5-883f-c551ac3fc79b None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:44.436528 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:44.467728 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffac07a-9b9b-448f-9ab0-7e46c56a86fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:44.468908 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffac07a-9b9b-448f-9ab0-7e46c56a86fc None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:44.469538 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffac07a-9b9b-448f-9ab0-7e46c56a86fc None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:44.471145 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffac07a-9b9b-448f-9ab0-7e46c56a86fc None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:44.472265 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffac07a-9b9b-448f-9ab0-7e46c56a86fc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:44.475787 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffac07a-9b9b-448f-9ab0-7e46c56a86fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:44.477549 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffac07a-9b9b-448f-9ab0-7e46c56a86fc None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:44.497867 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 199/410] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Jun 10 12:44:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:44.517333 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffac07a-9b9b-448f-9ab0-7e46c56a86fc None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:44.617502 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca247f0-ab60-43c5-bd7c-65abf7e29601 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:44.660889 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd6a726-cc66-42c7-ad67-f07d991a6ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:44.661666 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd6a726-cc66-42c7-ad67-f07d991a6ee7 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:44.662381 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd6a726-cc66-42c7-ad67-f07d991a6ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:44.662655 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd6a726-cc66-42c7-ad67-f07d991a6ee7 None admin] PATH_INFO: `/v3/projects/485e375bd3394086be6c5cd49bd19e2d` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:44.663530 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffac07a-9b9b-448f-9ab0-7e46c56a86fc None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:44.663911 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd6a726-cc66-42c7-ad67-f07d991a6ee7 None admin] RBAC: Authorizing `identity:get_project(project_id=485e375bd3394086be6c5cd49bd19e2d)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:44.665499 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd6a726-cc66-42c7-ad67-f07d991a6ee7 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:44.667681 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 212/411] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Jun 10 12:44:44 2022] GET /identity/v3/projects/485e375bd3394086be6c5cd49bd19e2d => generated 302 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:44.673325 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd6a726-cc66-42c7-ad67-f07d991a6ee7 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:44.704401 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:44.708509 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3646bdb0-1c13-417c-b235-746fd5643abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:44.709880 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3646bdb0-1c13-417c-b235-746fd5643abb None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:44.710598 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3646bdb0-1c13-417c-b235-746fd5643abb None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:44.711375 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3646bdb0-1c13-417c-b235-746fd5643abb None admin] PATH_INFO: `/v3/groups/52dd74368faf4035b19b7e1f005cf049` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:44.713060 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3646bdb0-1c13-417c-b235-746fd5643abb None admin] RBAC: Authorizing `identity:get_group(group_id=52dd74368faf4035b19b7e1f005cf049)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:44.715739 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3646bdb0-1c13-417c-b235-746fd5643abb None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:44.717841 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 200/412] 10.222.0.27 () {56 vars in 1180 bytes} [Fri Jun 10 12:44:44 2022] GET /identity/v3/groups/52dd74368faf4035b19b7e1f005cf049 => generated 229 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 12:44:44.720370 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d45da0-5482-4e9d-9149-525845bfd9c0 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:44.720370 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d45da0-5482-4e9d-9149-525845bfd9c0 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:44.720370 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d45da0-5482-4e9d-9149-525845bfd9c0 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:44.721221 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 201/413] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:44:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:44:44.726373 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d45da0-5482-4e9d-9149-525845bfd9c0 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:44.780842 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:44.789593 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd6a726-cc66-42c7-ad67-f07d991a6ee7 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:44.827075 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec1be9d-2424-43a4-bd32-a64fdb8793e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:44.827981 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec1be9d-2424-43a4-bd32-a64fdb8793e3 None admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:44.828263 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec1be9d-2424-43a4-bd32-a64fdb8793e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:44.828597 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec1be9d-2424-43a4-bd32-a64fdb8793e3 None admin] PATH_INFO: `/v3/projects/485e375bd3394086be6c5cd49bd19e2d/groups/ab486657b8d9476d9c728d34cfccfa66/roles/2d71ee7c2a544fe5ad77d92b24280efe` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:44.831087 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec1be9d-2424-43a4-bd32-a64fdb8793e3 None admin] RBAC: Authorizing `identity:create_grant(project_id=485e375bd3394086be6c5cd49bd19e2d, group_id=ab486657b8d9476d9c728d34cfccfa66, role_id=2d71ee7c2a544fe5ad77d92b24280efe)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:44.832627 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec1be9d-2424-43a4-bd32-a64fdb8793e3 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:44.854403 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d45da0-5482-4e9d-9149-525845bfd9c0 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:44.891267 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e430c92-200f-4be2-a881-dc6033a00291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:44.892238 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e430c92-200f-4be2-a881-dc6033a00291 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:44.892525 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e430c92-200f-4be2-a881-dc6033a00291 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:44.892900 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e430c92-200f-4be2-a881-dc6033a00291 None admin] PATH_INFO: `/v3/projects/2e3a9af772214953842e9cc63a52ae3a` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:44.894004 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e430c92-200f-4be2-a881-dc6033a00291 None admin] RBAC: Authorizing `identity:get_project(project_id=2e3a9af772214953842e9cc63a52ae3a)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:44.895428 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e430c92-200f-4be2-a881-dc6033a00291 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:44.901989 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 202/414] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Jun 10 12:44:44 2022] GET /identity/v3/projects/2e3a9af772214953842e9cc63a52ae3a => generated 344 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:44.901989 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413934cc-373c-4b6a-a063-1ab2ea4727c7 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:44.901989 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413934cc-373c-4b6a-a063-1ab2ea4727c7 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:44.901989 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413934cc-373c-4b6a-a063-1ab2ea4727c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:44.912516 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 213/415] 10.222.0.27 () {58 vars in 1442 bytes} [Fri Jun 10 12:44:44 2022] PUT /identity/v3/projects/485e375bd3394086be6c5cd49bd19e2d/groups/ab486657b8d9476d9c728d34cfccfa66/roles/2d71ee7c2a544fe5ad77d92b24280efe => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 12:44:44.913621 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec1be9d-2424-43a4-bd32-a64fdb8793e3 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:44.940816 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-413934cc-373c-4b6a-a063-1ab2ea4727c7 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:44:44.942168 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-413934cc-373c-4b6a-a063-1ab2ea4727c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:44:45.024318 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:45.039052 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec1be9d-2424-43a4-bd32-a64fdb8793e3 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:45.079501 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55190e7-07d6-4e29-8e0f-11addc904859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:45.080787 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55190e7-07d6-4e29-8e0f-11addc904859 None admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:45.081231 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55190e7-07d6-4e29-8e0f-11addc904859 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:45.081641 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55190e7-07d6-4e29-8e0f-11addc904859 None admin] PATH_INFO: `/v3/projects/2e3a9af772214953842e9cc63a52ae3a/groups/52dd74368faf4035b19b7e1f005cf049/roles/b43d5a8247e244869c2c70f986f42e8a` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:45.084230 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55190e7-07d6-4e29-8e0f-11addc904859 None admin] RBAC: Authorizing `identity:create_grant(project_id=2e3a9af772214953842e9cc63a52ae3a, group_id=52dd74368faf4035b19b7e1f005cf049, role_id=b43d5a8247e244869c2c70f986f42e8a)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:45.087002 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55190e7-07d6-4e29-8e0f-11addc904859 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:45.135318 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 203/416] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:44:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 10 12:44:45.160314 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 214/417] 10.222.0.27 () {58 vars in 1442 bytes} [Fri Jun 10 12:44:44 2022] PUT /identity/v3/projects/2e3a9af772214953842e9cc63a52ae3a/groups/52dd74368faf4035b19b7e1f005cf049/roles/b43d5a8247e244869c2c70f986f42e8a => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 12:44:45.168042 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413934cc-373c-4b6a-a063-1ab2ea4727c7 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:45.204487 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:45.279796 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413934cc-373c-4b6a-a063-1ab2ea4727c7 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:45.322731 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf578b0-1588-45f6-894d-e74703d8fbc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:45.323552 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf578b0-1588-45f6-894d-e74703d8fbc7 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:45.323838 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf578b0-1588-45f6-894d-e74703d8fbc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:45.324137 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf578b0-1588-45f6-894d-e74703d8fbc7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:45.344335 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf578b0-1588-45f6-894d-e74703d8fbc7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:44:45.352344 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-9cf578b0-1588-45f6-894d-e74703d8fbc7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 10 12:44:45.352344 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:44:45.352344 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:44:45.352344 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:44:45.352344 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:44:45.352344 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:44:45.352344 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:44:45.352344 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:44:45.352344 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:44:45.352344 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:44:45.352344 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:44:45.352344 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:44:45.352344 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:44:45.352344 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:44:45.352344 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:44:45.352344 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:44:45.353334 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:44:45.353334 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:44:45.353334 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:44:45.353334 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:44:45.353334 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:44:45.353334 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:44:45.353334 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:44:45.353334 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:44:45.353334 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:44:45.353334 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:44:45.353334 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:44:45.353334 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:44:45.353334 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:44:45.353334 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:44:45.353334 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:44:45.353334 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:44:45.354148 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:44:45.354148 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:44:45.354148 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:44:45.354148 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:44:45.354148 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:44:45.354148 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:44:45.354148 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:44:45.354148 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:44:45.354148 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:44:45.354148 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:44:45.354148 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 10 12:44:45.354148 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:44:45.354148 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 204/418] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Jun 10 12:44:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:44:45.359769 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55190e7-07d6-4e29-8e0f-11addc904859 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:45.503919 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55190e7-07d6-4e29-8e0f-11addc904859 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:45.540207 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a03ac6-95f4-4e4e-a8cc-d90787c31c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:45.540888 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a03ac6-95f4-4e4e-a8cc-d90787c31c8e None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:45.541174 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a03ac6-95f4-4e4e-a8cc-d90787c31c8e None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:45.541456 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a03ac6-95f4-4e4e-a8cc-d90787c31c8e None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:45.542218 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a03ac6-95f4-4e4e-a8cc-d90787c31c8e None admin] RBAC: Adding query filter params (name=member) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:45.542529 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a03ac6-95f4-4e4e-a8cc-d90787c31c8e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:45.545437 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a03ac6-95f4-4e4e-a8cc-d90787c31c8e None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:45.555230 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 215/419] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Jun 10 12:44:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:45.561101 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf578b0-1588-45f6-894d-e74703d8fbc7 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:45.633147 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183c5f89-56eb-4d3f-ac63-fe99caa71c18 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:45.633482 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183c5f89-56eb-4d3f-ac63-fe99caa71c18 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:45.633710 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183c5f89-56eb-4d3f-ac63-fe99caa71c18 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:45.635911 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 216/420] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:44:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:44:45.640927 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba11b76-3c86-46d5-b85c-3fc0e8e8c9b5 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:45.641216 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba11b76-3c86-46d5-b85c-3fc0e8e8c9b5 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:45.641509 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba11b76-3c86-46d5-b85c-3fc0e8e8c9b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:45.653487 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:45.658619 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-aba11b76-3c86-46d5-b85c-3fc0e8e8c9b5 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:44:45.659827 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-aba11b76-3c86-46d5-b85c-3fc0e8e8c9b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:44:45.678135 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:45.686158 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf578b0-1588-45f6-894d-e74703d8fbc7 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:45.742883 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f4ee6d-4625-48ee-807f-a9fee8c4de3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:45.745294 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f4ee6d-4625-48ee-807f-a9fee8c4de3e None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:45.745712 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f4ee6d-4625-48ee-807f-a9fee8c4de3e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:45.745995 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f4ee6d-4625-48ee-807f-a9fee8c4de3e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:45.751039 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 217/421] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:44:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 10 12:44:45.812425 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba11b76-3c86-46d5-b85c-3fc0e8e8c9b5 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:45.844693 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f4ee6d-4625-48ee-807f-a9fee8c4de3e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:45.846482 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f4ee6d-4625-48ee-807f-a9fee8c4de3e None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:45.939813 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba11b76-3c86-46d5-b85c-3fc0e8e8c9b5 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:45.996133 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fafc9af-6722-4153-926c-16b829d05029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:45.997339 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fafc9af-6722-4153-926c-16b829d05029 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:45.997827 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fafc9af-6722-4153-926c-16b829d05029 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:45.998316 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fafc9af-6722-4153-926c-16b829d05029 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:46.004166 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:46.021384 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fafc9af-6722-4153-926c-16b829d05029 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:44:46.037342 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-8fafc9af-6722-4153-926c-16b829d05029 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 10 12:44:46.037342 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:44:46.037342 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:44:46.037342 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:44:46.037342 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:44:46.037342 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:44:46.037342 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:44:46.037342 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:44:46.037342 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:44:46.037342 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:44:46.037342 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:44:46.037342 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:44:46.037342 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:44:46.037342 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:44:46.037342 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:44:46.037342 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:44:46.038266 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:44:46.038266 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:44:46.038266 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:44:46.038266 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:44:46.038266 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:44:46.038266 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:44:46.038266 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:44:46.038266 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:44:46.038266 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:44:46.038266 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:44:46.038266 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:44:46.038266 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:44:46.038266 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:44:46.038266 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:44:46.038266 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:44:46.038266 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:44:46.039492 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:44:46.039492 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:44:46.039492 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:44:46.039492 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:44:46.039492 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:44:46.039492 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:44:46.039492 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:44:46.039492 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:44:46.039492 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:44:46.039492 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:44:46.039492 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 10 12:44:46.039492 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:44:46.039492 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 218/422] 10.222.0.27 () {56 vars in 1114 bytes} [Fri Jun 10 12:44:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 10 12:44:46.047959 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:46.053840 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fafc9af-6722-4153-926c-16b829d05029 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:46.057316 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 205/423] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Jun 10 12:44:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 10 12:44:46.071000 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f4ee6d-4625-48ee-807f-a9fee8c4de3e None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:46.179503 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f4ee6d-4625-48ee-807f-a9fee8c4de3e None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:46.216929 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787b29f1-519c-4c9c-9846-b4b23b96e6e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:46.217802 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787b29f1-519c-4c9c-9846-b4b23b96e6e9 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:46.218094 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787b29f1-519c-4c9c-9846-b4b23b96e6e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:46.218378 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787b29f1-519c-4c9c-9846-b4b23b96e6e9 None admin] PATH_INFO: `/v3/projects/3663edc029e8431d904a228a9e5755a0` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:46.219739 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787b29f1-519c-4c9c-9846-b4b23b96e6e9 None admin] RBAC: Authorizing `identity:get_project(project_id=3663edc029e8431d904a228a9e5755a0)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:46.221240 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787b29f1-519c-4c9c-9846-b4b23b96e6e9 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:46.223309 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 206/424] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Jun 10 12:44:46 2022] GET /identity/v3/projects/3663edc029e8431d904a228a9e5755a0 => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 12:44:46.231697 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787b29f1-519c-4c9c-9846-b4b23b96e6e9 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:46.247947 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fafc9af-6722-4153-926c-16b829d05029 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:46.315387 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22014ccf-df5b-40fb-9913-fa39e5b1732f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:46.316085 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22014ccf-df5b-40fb-9913-fa39e5b1732f None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:46.316372 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22014ccf-df5b-40fb-9913-fa39e5b1732f None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:46.316649 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22014ccf-df5b-40fb-9913-fa39e5b1732f None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:46.317403 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22014ccf-df5b-40fb-9913-fa39e5b1732f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:46.317712 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22014ccf-df5b-40fb-9913-fa39e5b1732f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:46.319110 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22014ccf-df5b-40fb-9913-fa39e5b1732f None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:46.335897 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 219/425] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Jun 10 12:44:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 12:44:46.343952 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22014ccf-df5b-40fb-9913-fa39e5b1732f None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:46.346950 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:46.387887 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787b29f1-519c-4c9c-9846-b4b23b96e6e9 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:46.427892 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b117295e-4a02-4e19-a833-ca980753b015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:46.427892 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b117295e-4a02-4e19-a833-ca980753b015 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:46.427892 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b117295e-4a02-4e19-a833-ca980753b015 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:46.427892 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b117295e-4a02-4e19-a833-ca980753b015 None admin] PATH_INFO: `/v3/groups/ab486657b8d9476d9c728d34cfccfa66` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:46.428672 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b117295e-4a02-4e19-a833-ca980753b015 None admin] RBAC: Authorizing `identity:get_group(group_id=ab486657b8d9476d9c728d34cfccfa66)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:46.428672 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b117295e-4a02-4e19-a833-ca980753b015 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:46.432817 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 207/426] 10.222.0.27 () {56 vars in 1180 bytes} [Fri Jun 10 12:44:46 2022] GET /identity/v3/groups/ab486657b8d9476d9c728d34cfccfa66 => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:46.439883 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b117295e-4a02-4e19-a833-ca980753b015 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:46.462178 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:46.546586 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22014ccf-df5b-40fb-9913-fa39e5b1732f None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:46.599185 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b117295e-4a02-4e19-a833-ca980753b015 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:46.615219 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad43478a-75a2-4d0a-8e74-e39dff2c926c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:46.623910 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad43478a-75a2-4d0a-8e74-e39dff2c926c None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:46.624251 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad43478a-75a2-4d0a-8e74-e39dff2c926c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:46.624544 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad43478a-75a2-4d0a-8e74-e39dff2c926c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:46.630598 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:46.635662 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675774df-6ca5-470a-aa50-a13a0744a299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:46.636352 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675774df-6ca5-470a-aa50-a13a0744a299 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:46.636637 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675774df-6ca5-470a-aa50-a13a0744a299 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:46.636918 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675774df-6ca5-470a-aa50-a13a0744a299 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:46.637758 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675774df-6ca5-470a-aa50-a13a0744a299 None admin] RBAC: Adding query filter params (group.id=ab486657b8d9476d9c728d34cfccfa66, role.id=43106c664e1b40de984c0c62701149c8, scope.project.id=3663edc029e8431d904a228a9e5755a0) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:46.638074 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675774df-6ca5-470a-aa50-a13a0744a299 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:46.640152 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675774df-6ca5-470a-aa50-a13a0744a299 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:46.649978 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 208/427] 10.222.0.27 () {56 vars in 1376 bytes} [Fri Jun 10 12:44:46 2022] GET /identity/v3/role_assignments?group.id=ab486657b8d9476d9c728d34cfccfa66&scope.project.id=3663edc029e8431d904a228a9e5755a0&role.id=43106c664e1b40de984c0c62701149c8 => generated 611 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:46.748384 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad43478a-75a2-4d0a-8e74-e39dff2c926c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:46.749859 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad43478a-75a2-4d0a-8e74-e39dff2c926c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:46.812507 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:46.859470 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 220/428] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Jun 10 12:44:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 519 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 10 12:44:46.866433 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675774df-6ca5-470a-aa50-a13a0744a299 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:46.907522 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ef2f92-982e-49df-b579-d75f6c8cef9d None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:46.908184 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ef2f92-982e-49df-b579-d75f6c8cef9d None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:46.908671 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ef2f92-982e-49df-b579-d75f6c8cef9d None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:46.911878 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 221/429] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:44:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 10 12:44:46.917058 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f140b464-53f9-466c-a778-ae9030e5d991 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:46.917577 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f140b464-53f9-466c-a778-ae9030e5d991 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:46.918049 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f140b464-53f9-466c-a778-ae9030e5d991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:46.937333 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-f140b464-53f9-466c-a778-ae9030e5d991 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:44:46.939074 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f140b464-53f9-466c-a778-ae9030e5d991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:44:46.976778 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675774df-6ca5-470a-aa50-a13a0744a299 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:47.023849 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94c9b99-acc5-4246-be44-766d6874f9a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:47.025386 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94c9b99-acc5-4246-be44-766d6874f9a8 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:47.025667 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94c9b99-acc5-4246-be44-766d6874f9a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:47.025938 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94c9b99-acc5-4246-be44-766d6874f9a8 None admin] PATH_INFO: `/v3/projects/3663edc029e8431d904a228a9e5755a0` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:47.027377 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94c9b99-acc5-4246-be44-766d6874f9a8 None admin] RBAC: Authorizing `identity:get_project(project_id=3663edc029e8431d904a228a9e5755a0)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:47.029957 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94c9b99-acc5-4246-be44-766d6874f9a8 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:47.030706 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 209/430] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Jun 10 12:44:46 2022] GET /identity/v3/projects/3663edc029e8431d904a228a9e5755a0 => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:47.036529 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 222/431] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:44:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 10 12:44:47.037692 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94c9b99-acc5-4246-be44-766d6874f9a8 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:47.063705 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f140b464-53f9-466c-a778-ae9030e5d991 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:47.118923 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:47.167329 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94c9b99-acc5-4246-be44-766d6874f9a8 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:47.188297 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:47.204051 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f140b464-53f9-466c-a778-ae9030e5d991 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:47.208493 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19480ce-1df9-4413-9b7f-478f84ce6b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:47.209514 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19480ce-1df9-4413-9b7f-478f84ce6b47 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:47.209908 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19480ce-1df9-4413-9b7f-478f84ce6b47 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:47.210287 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19480ce-1df9-4413-9b7f-478f84ce6b47 None admin] PATH_INFO: `/v3/groups/ab486657b8d9476d9c728d34cfccfa66` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:47.211602 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19480ce-1df9-4413-9b7f-478f84ce6b47 None admin] RBAC: Authorizing `identity:get_group(group_id=ab486657b8d9476d9c728d34cfccfa66)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:47.213819 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19480ce-1df9-4413-9b7f-478f84ce6b47 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:47.216629 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 210/432] 10.222.0.27 () {56 vars in 1180 bytes} [Fri Jun 10 12:44:47 2022] GET /identity/v3/groups/ab486657b8d9476d9c728d34cfccfa66 => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:47.222096 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19480ce-1df9-4413-9b7f-478f84ce6b47 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:47.254248 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e5241f-d3f6-46f3-bcf5-a3316cd2dc7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:47.255205 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e5241f-d3f6-46f3-bcf5-a3316cd2dc7a None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:47.255591 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e5241f-d3f6-46f3-bcf5-a3316cd2dc7a None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:47.256009 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e5241f-d3f6-46f3-bcf5-a3316cd2dc7a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:47.268972 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e5241f-d3f6-46f3-bcf5-a3316cd2dc7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:44:47.277735 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-c1e5241f-d3f6-46f3-bcf5-a3316cd2dc7a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 10 12:44:47.277735 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:44:47.277735 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:44:47.277735 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:44:47.277735 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:44:47.277735 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:44:47.277735 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:44:47.277735 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:44:47.277735 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:44:47.277735 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:44:47.277735 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:44:47.277735 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:44:47.277735 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:44:47.277735 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:44:47.277735 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:44:47.277735 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:44:47.278660 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:44:47.278660 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:44:47.278660 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:44:47.278660 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:44:47.278660 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:44:47.278660 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:44:47.278660 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:44:47.278660 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:44:47.278660 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:44:47.278660 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:44:47.278660 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:44:47.278660 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:44:47.278660 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:44:47.278660 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:44:47.278660 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:44:47.278660 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:44:47.279460 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:44:47.279460 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:44:47.279460 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:44:47.279460 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:44:47.279460 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:44:47.279460 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:44:47.279460 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:44:47.279460 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:44:47.279460 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:44:47.279460 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:44:47.279460 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 10 12:44:47.279460 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:44:47.280587 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 223/433] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Jun 10 12:44:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 10 12:44:47.285981 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e5241f-d3f6-46f3-bcf5-a3316cd2dc7a None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:47.356325 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19480ce-1df9-4413-9b7f-478f84ce6b47 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:47.395562 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:47.401012 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e5241f-d3f6-46f3-bcf5-a3316cd2dc7a None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:47.401962 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97c031d-c5be-41ad-8716-d167633a6713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:47.402628 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97c031d-c5be-41ad-8716-d167633a6713 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:47.402908 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97c031d-c5be-41ad-8716-d167633a6713 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:47.403184 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97c031d-c5be-41ad-8716-d167633a6713 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:47.404067 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97c031d-c5be-41ad-8716-d167633a6713 None admin] RBAC: Adding query filter params (group.id=ab486657b8d9476d9c728d34cfccfa66, role.id=2d71ee7c2a544fe5ad77d92b24280efe, scope.project.id=3663edc029e8431d904a228a9e5755a0) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:47.404377 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97c031d-c5be-41ad-8716-d167633a6713 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:47.407050 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97c031d-c5be-41ad-8716-d167633a6713 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:47.414814 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 211/434] 10.222.0.27 () {56 vars in 1376 bytes} [Fri Jun 10 12:44:47 2022] GET /identity/v3/role_assignments?group.id=ab486657b8d9476d9c728d34cfccfa66&scope.project.id=3663edc029e8431d904a228a9e5755a0&role.id=2d71ee7c2a544fe5ad77d92b24280efe => generated 611 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 12:44:47.448591 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd80af03-87ef-468f-9723-bea779b138dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:47.449284 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd80af03-87ef-468f-9723-bea779b138dd None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:47.449571 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd80af03-87ef-468f-9723-bea779b138dd None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:47.449852 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd80af03-87ef-468f-9723-bea779b138dd None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:47.450625 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd80af03-87ef-468f-9723-bea779b138dd None admin] RBAC: Adding query filter params (name=member) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:47.450937 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd80af03-87ef-468f-9723-bea779b138dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:47.452425 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd80af03-87ef-468f-9723-bea779b138dd None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:47.463590 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 224/435] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Jun 10 12:44:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 12:44:47.469258 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97c031d-c5be-41ad-8716-d167633a6713 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:47.597191 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97c031d-c5be-41ad-8716-d167633a6713 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:47.638465 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfa7f6d-36e0-4701-9dfa-5ef2d6e87afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:47.639420 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa7f6d-36e0-4701-9dfa-5ef2d6e87afd None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:47.639945 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa7f6d-36e0-4701-9dfa-5ef2d6e87afd None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:47.641933 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8239068a-79a2-4ee3-87bf-d07033450023 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:47.642383 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8239068a-79a2-4ee3-87bf-d07033450023 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:47.642664 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8239068a-79a2-4ee3-87bf-d07033450023 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:47.642931 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa7f6d-36e0-4701-9dfa-5ef2d6e87afd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:47.644340 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 225/436] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:44:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 10 12:44:47.650100 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937f5924-edf3-4ccf-90f6-67a67a675c2b None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:47.650331 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937f5924-edf3-4ccf-90f6-67a67a675c2b None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:47.650596 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937f5924-edf3-4ccf-90f6-67a67a675c2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:47.673860 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-937f5924-edf3-4ccf-90f6-67a67a675c2b None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:44:47.675054 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-937f5924-edf3-4ccf-90f6-67a67a675c2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:44:47.678151 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:47.719734 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:47.728634 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfa7f6d-36e0-4701-9dfa-5ef2d6e87afd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:47.730681 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfa7f6d-36e0-4701-9dfa-5ef2d6e87afd None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:47.757739 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 226/437] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:44:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 10 12:44:47.781227 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937f5924-edf3-4ccf-90f6-67a67a675c2b None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:47.859905 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 212/438] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Jun 10 12:44:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 10 12:44:47.865309 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfa7f6d-36e0-4701-9dfa-5ef2d6e87afd None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:47.918028 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937f5924-edf3-4ccf-90f6-67a67a675c2b None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:47.955394 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f50e7d9-ffd2-49ad-a72a-71db09275367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:47.956339 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f50e7d9-ffd2-49ad-a72a-71db09275367 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:47.956625 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f50e7d9-ffd2-49ad-a72a-71db09275367 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:47.956904 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f50e7d9-ffd2-49ad-a72a-71db09275367 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:47.964306 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:47.967224 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f50e7d9-ffd2-49ad-a72a-71db09275367 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:44:47.973183 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:47.975398 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-8f50e7d9-ffd2-49ad-a72a-71db09275367 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 10 12:44:47.975398 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:44:47.975398 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:44:47.975398 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:44:47.975398 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:44:47.975398 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:44:47.975398 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:44:47.975398 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:44:47.975398 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:44:47.975398 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:44:47.975398 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:44:47.975398 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:44:47.975398 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:44:47.975398 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:44:47.975398 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:44:47.975398 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:44:47.988943 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:44:47.988943 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:44:47.988943 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:44:47.988943 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:44:47.988943 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:44:47.988943 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:44:47.988943 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:44:47.988943 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:44:47.988943 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:44:47.988943 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:44:47.988943 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:44:47.988943 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:44:47.988943 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:44:47.988943 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:44:47.988943 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:44:47.988943 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:44:47.990118 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:44:47.990118 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:44:47.990118 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:44:47.990118 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:44:47.990118 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:44:47.990118 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:44:47.990118 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:44:47.990118 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:44:47.990118 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:44:47.990118 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:44:47.990118 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 10 12:44:47.990118 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:44:47.990118 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 227/439] 10.222.0.27 () {56 vars in 1114 bytes} [Fri Jun 10 12:44:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:44:47.990118 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4af115a-8b8d-4694-bb89-d65598761c4f None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:47.990850 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4af115a-8b8d-4694-bb89-d65598761c4f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:47.990850 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4af115a-8b8d-4694-bb89-d65598761c4f None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:47.990850 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 228/440] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:44:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:44:47.991031 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfa7f6d-36e0-4701-9dfa-5ef2d6e87afd None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:47.991104 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4af115a-8b8d-4694-bb89-d65598761c4f None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:48.017019 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be39f168-db36-4500-8e3a-ac9857cba553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:48.017872 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be39f168-db36-4500-8e3a-ac9857cba553 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:48.018161 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be39f168-db36-4500-8e3a-ac9857cba553 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:48.018434 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be39f168-db36-4500-8e3a-ac9857cba553 None admin] PATH_INFO: `/v3/projects/485e375bd3394086be6c5cd49bd19e2d` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:48.019613 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be39f168-db36-4500-8e3a-ac9857cba553 None admin] RBAC: Authorizing `identity:get_project(project_id=485e375bd3394086be6c5cd49bd19e2d)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:48.021099 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be39f168-db36-4500-8e3a-ac9857cba553 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:48.022779 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 213/441] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Jun 10 12:44:47 2022] GET /identity/v3/projects/485e375bd3394086be6c5cd49bd19e2d => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:48.023917 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932c9133-87a3-4316-ac03-553b63879190 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:48.024207 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932c9133-87a3-4316-ac03-553b63879190 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:48.024514 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932c9133-87a3-4316-ac03-553b63879190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:48.041929 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-932c9133-87a3-4316-ac03-553b63879190 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:44:48.043129 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-932c9133-87a3-4316-ac03-553b63879190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:44:48.083513 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4af115a-8b8d-4694-bb89-d65598761c4f None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:48.107769 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 214/442] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:44:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 10 12:44:48.108629 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932c9133-87a3-4316-ac03-553b63879190 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:48.116257 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b728de09-98aa-4a79-8fd2-84ebaab157fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:48.116915 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b728de09-98aa-4a79-8fd2-84ebaab157fd None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:48.117194 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b728de09-98aa-4a79-8fd2-84ebaab157fd None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:48.117466 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b728de09-98aa-4a79-8fd2-84ebaab157fd None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:48.118268 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b728de09-98aa-4a79-8fd2-84ebaab157fd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:48.119081 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b728de09-98aa-4a79-8fd2-84ebaab157fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:48.120484 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b728de09-98aa-4a79-8fd2-84ebaab157fd None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:48.128273 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 229/443] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Jun 10 12:44:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 12:44:48.131289 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b728de09-98aa-4a79-8fd2-84ebaab157fd None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:48.190587 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:48.215155 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:48.221498 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932c9133-87a3-4316-ac03-553b63879190 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:48.228627 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b728de09-98aa-4a79-8fd2-84ebaab157fd None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:48.254788 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e151d69-cff5-4af1-bfe1-c9e7ca27396e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:48.255635 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e151d69-cff5-4af1-bfe1-c9e7ca27396e None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:48.255943 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e151d69-cff5-4af1-bfe1-c9e7ca27396e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:48.256218 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e151d69-cff5-4af1-bfe1-c9e7ca27396e None admin] PATH_INFO: `/v3/groups/ab486657b8d9476d9c728d34cfccfa66` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:48.257372 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e151d69-cff5-4af1-bfe1-c9e7ca27396e None admin] RBAC: Authorizing `identity:get_group(group_id=ab486657b8d9476d9c728d34cfccfa66)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:48.258778 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e151d69-cff5-4af1-bfe1-c9e7ca27396e None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:48.260398 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 215/444] 10.222.0.27 () {56 vars in 1180 bytes} [Fri Jun 10 12:44:48 2022] GET /identity/v3/groups/ab486657b8d9476d9c728d34cfccfa66 => generated 226 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:48.261136 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e151d69-cff5-4af1-bfe1-c9e7ca27396e None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:48.261247 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5210862-5bbc-47c6-931a-b46a31b504ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:48.262029 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5210862-5bbc-47c6-931a-b46a31b504ea None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:48.262316 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5210862-5bbc-47c6-931a-b46a31b504ea None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:48.262590 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5210862-5bbc-47c6-931a-b46a31b504ea None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:48.273167 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5210862-5bbc-47c6-931a-b46a31b504ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:44:48.280737 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-c5210862-5bbc-47c6-931a-b46a31b504ea None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 10 12:44:48.280737 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:44:48.280737 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:44:48.280737 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:44:48.280737 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:44:48.280737 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:44:48.280737 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:44:48.280737 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:44:48.280737 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:44:48.280737 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:44:48.280737 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:44:48.280737 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:44:48.280737 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:44:48.280737 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:44:48.280737 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:44:48.280737 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:44:48.281652 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:44:48.281652 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:44:48.281652 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:44:48.281652 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:44:48.281652 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:44:48.281652 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:44:48.281652 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:44:48.281652 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:44:48.281652 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:44:48.281652 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:44:48.281652 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:44:48.281652 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:44:48.281652 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:44:48.281652 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:44:48.281652 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:44:48.281652 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:44:48.282480 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:44:48.282480 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:44:48.282480 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:44:48.282480 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:44:48.282480 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:44:48.282480 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:44:48.282480 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:44:48.282480 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:44:48.282480 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:44:48.282480 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:44:48.282480 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 10 12:44:48.282480 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:44:48.282480 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 230/445] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Jun 10 12:44:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 10 12:44:48.283143 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5210862-5bbc-47c6-931a-b46a31b504ea None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:48.360276 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e151d69-cff5-4af1-bfe1-c9e7ca27396e None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:48.381796 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5210862-5bbc-47c6-931a-b46a31b504ea None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:48.393255 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764c6b80-eddc-4566-8212-7962fbd4c678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:48.393904 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764c6b80-eddc-4566-8212-7962fbd4c678 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:48.394184 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764c6b80-eddc-4566-8212-7962fbd4c678 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:48.394460 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764c6b80-eddc-4566-8212-7962fbd4c678 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:48.403654 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:48.415085 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb948feb-5146-449d-9c27-3199a6ede9a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:48.416264 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb948feb-5146-449d-9c27-3199a6ede9a5 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:48.417429 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb948feb-5146-449d-9c27-3199a6ede9a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:48.417429 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb948feb-5146-449d-9c27-3199a6ede9a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:48.417725 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb948feb-5146-449d-9c27-3199a6ede9a5 None admin] RBAC: Adding query filter params (group.id=ab486657b8d9476d9c728d34cfccfa66, role.id=43106c664e1b40de984c0c62701149c8, scope.project.id=485e375bd3394086be6c5cd49bd19e2d) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:48.418036 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb948feb-5146-449d-9c27-3199a6ede9a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:48.419467 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb948feb-5146-449d-9c27-3199a6ede9a5 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:48.424475 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:48.425510 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 231/446] 10.222.0.27 () {56 vars in 1376 bytes} [Fri Jun 10 12:44:48 2022] GET /identity/v3/role_assignments?group.id=ab486657b8d9476d9c728d34cfccfa66&scope.project.id=485e375bd3394086be6c5cd49bd19e2d&role.id=43106c664e1b40de984c0c62701149c8 => generated 611 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:48.426285 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb948feb-5146-449d-9c27-3199a6ede9a5 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:48.468546 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764c6b80-eddc-4566-8212-7962fbd4c678 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:48.470184 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764c6b80-eddc-4566-8212-7962fbd4c678 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:48.546315 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb948feb-5146-449d-9c27-3199a6ede9a5 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:48.583216 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbb2c97-20af-4a44-881f-41caf49c10cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:48.583925 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbb2c97-20af-4a44-881f-41caf49c10cd None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:48.584215 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbb2c97-20af-4a44-881f-41caf49c10cd None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:48.585129 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbb2c97-20af-4a44-881f-41caf49c10cd None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:48.585234 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbb2c97-20af-4a44-881f-41caf49c10cd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:48.585615 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbb2c97-20af-4a44-881f-41caf49c10cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:48.587915 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbb2c97-20af-4a44-881f-41caf49c10cd None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:48.597194 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 232/447] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Jun 10 12:44:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:48.603971 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbb2c97-20af-4a44-881f-41caf49c10cd None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:48.606306 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 216/448] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Jun 10 12:44:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 10 12:44:48.611600 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764c6b80-eddc-4566-8212-7962fbd4c678 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:48.657743 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:48.682032 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:48.717789 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbb2c97-20af-4a44-881f-41caf49c10cd None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:48.728226 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764c6b80-eddc-4566-8212-7962fbd4c678 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:48.762707 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346ad613-daaf-4a72-a663-1eb5e08c6b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:48.763365 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346ad613-daaf-4a72-a663-1eb5e08c6b88 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:48.763650 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346ad613-daaf-4a72-a663-1eb5e08c6b88 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:48.763957 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346ad613-daaf-4a72-a663-1eb5e08c6b88 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:48.766499 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea67141-c11d-4952-a962-706e9a165aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:48.767394 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea67141-c11d-4952-a962-706e9a165aca None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:48.767681 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea67141-c11d-4952-a962-706e9a165aca None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:48.767991 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea67141-c11d-4952-a962-706e9a165aca None admin] PATH_INFO: `/v3/projects/485e375bd3394086be6c5cd49bd19e2d` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:48.769199 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea67141-c11d-4952-a962-706e9a165aca None admin] RBAC: Authorizing `identity:get_project(project_id=485e375bd3394086be6c5cd49bd19e2d)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:48.770653 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea67141-c11d-4952-a962-706e9a165aca None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:48.772449 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 217/449] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Jun 10 12:44:48 2022] GET /identity/v3/projects/485e375bd3394086be6c5cd49bd19e2d => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 12:44:48.779118 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea67141-c11d-4952-a962-706e9a165aca None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:48.834469 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346ad613-daaf-4a72-a663-1eb5e08c6b88 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:48.835960 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346ad613-daaf-4a72-a663-1eb5e08c6b88 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:48.882739 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:48.887034 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea67141-c11d-4952-a962-706e9a165aca None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:48.916496 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:48.922321 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce982db1-4e07-4b77-ade1-9372a15244e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:48.923179 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce982db1-4e07-4b77-ade1-9372a15244e1 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:48.923463 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce982db1-4e07-4b77-ade1-9372a15244e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:48.925106 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce982db1-4e07-4b77-ade1-9372a15244e1 None admin] PATH_INFO: `/v3/groups/ab486657b8d9476d9c728d34cfccfa66` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:48.925106 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce982db1-4e07-4b77-ade1-9372a15244e1 None admin] RBAC: Authorizing `identity:get_group(group_id=ab486657b8d9476d9c728d34cfccfa66)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:48.926610 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce982db1-4e07-4b77-ade1-9372a15244e1 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:48.928269 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 218/450] 10.222.0.27 () {56 vars in 1180 bytes} [Fri Jun 10 12:44:48 2022] GET /identity/v3/groups/ab486657b8d9476d9c728d34cfccfa66 => generated 226 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:48.934179 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce982db1-4e07-4b77-ade1-9372a15244e1 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:48.940186 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 233/451] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Jun 10 12:44:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 10 12:44:48.950291 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346ad613-daaf-4a72-a663-1eb5e08c6b88 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:49.034755 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce982db1-4e07-4b77-ade1-9372a15244e1 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:49.054052 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346ad613-daaf-4a72-a663-1eb5e08c6b88 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:49.066743 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b758b4-97cc-4ffc-9b97-472b20ef2945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:49.067423 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b758b4-97cc-4ffc-9b97-472b20ef2945 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:49.067711 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b758b4-97cc-4ffc-9b97-472b20ef2945 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:49.068020 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b758b4-97cc-4ffc-9b97-472b20ef2945 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:49.068848 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b758b4-97cc-4ffc-9b97-472b20ef2945 None admin] RBAC: Adding query filter params (group.id=ab486657b8d9476d9c728d34cfccfa66, role.id=2d71ee7c2a544fe5ad77d92b24280efe, scope.project.id=485e375bd3394086be6c5cd49bd19e2d) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:49.069160 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b758b4-97cc-4ffc-9b97-472b20ef2945 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:49.070579 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b758b4-97cc-4ffc-9b97-472b20ef2945 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:49.077145 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 219/452] 10.222.0.27 () {56 vars in 1376 bytes} [Fri Jun 10 12:44:48 2022] GET /identity/v3/role_assignments?group.id=ab486657b8d9476d9c728d34cfccfa66&scope.project.id=485e375bd3394086be6c5cd49bd19e2d&role.id=2d71ee7c2a544fe5ad77d92b24280efe => generated 611 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 12:44:49.087624 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0bed90-d173-4afd-80d1-61ccaf386c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:49.088609 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0bed90-d173-4afd-80d1-61ccaf386c92 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:49.088893 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0bed90-d173-4afd-80d1-61ccaf386c92 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:49.089166 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0bed90-d173-4afd-80d1-61ccaf386c92 None admin] PATH_INFO: `/v3/projects/2e3a9af772214953842e9cc63a52ae3a` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:49.090434 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0bed90-d173-4afd-80d1-61ccaf386c92 None admin] RBAC: Authorizing `identity:get_project(project_id=2e3a9af772214953842e9cc63a52ae3a)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:49.091965 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0bed90-d173-4afd-80d1-61ccaf386c92 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:49.094083 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 234/453] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Jun 10 12:44:48 2022] GET /identity/v3/projects/2e3a9af772214953842e9cc63a52ae3a => generated 344 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 12:44:49.099421 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b758b4-97cc-4ffc-9b97-472b20ef2945 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:49.168567 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:49.208783 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b758b4-97cc-4ffc-9b97-472b20ef2945 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:49.244318 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1f0acb-e345-4d88-86a7-05c2733e97d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:49.245538 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1f0acb-e345-4d88-86a7-05c2733e97d8 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:49.245944 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1f0acb-e345-4d88-86a7-05c2733e97d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:49.246350 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1f0acb-e345-4d88-86a7-05c2733e97d8 None admin] PATH_INFO: `/v3/groups/52dd74368faf4035b19b7e1f005cf049` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:49.247717 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1f0acb-e345-4d88-86a7-05c2733e97d8 None admin] RBAC: Authorizing `identity:get_group(group_id=52dd74368faf4035b19b7e1f005cf049)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:49.249259 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1f0acb-e345-4d88-86a7-05c2733e97d8 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:49.250886 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 220/454] 10.222.0.27 () {56 vars in 1180 bytes} [Fri Jun 10 12:44:49 2022] GET /identity/v3/groups/52dd74368faf4035b19b7e1f005cf049 => generated 229 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:49.256686 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0bed90-d173-4afd-80d1-61ccaf386c92 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:49.277774 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:49.359225 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0bed90-d173-4afd-80d1-61ccaf386c92 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:49.393158 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d64e097-fe9b-4d61-b1d0-91a43a0bf5cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:49.393803 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d64e097-fe9b-4d61-b1d0-91a43a0bf5cf None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:49.394082 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d64e097-fe9b-4d61-b1d0-91a43a0bf5cf None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:49.394352 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d64e097-fe9b-4d61-b1d0-91a43a0bf5cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:49.395176 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d64e097-fe9b-4d61-b1d0-91a43a0bf5cf None admin] RBAC: Adding query filter params (group.id=52dd74368faf4035b19b7e1f005cf049, role.id=b43d5a8247e244869c2c70f986f42e8a, scope.project.id=2e3a9af772214953842e9cc63a52ae3a) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:49.395482 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d64e097-fe9b-4d61-b1d0-91a43a0bf5cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:49.396925 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d64e097-fe9b-4d61-b1d0-91a43a0bf5cf None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:49.402745 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 235/455] 10.222.0.27 () {56 vars in 1376 bytes} [Fri Jun 10 12:44:49 2022] GET /identity/v3/role_assignments?group.id=52dd74368faf4035b19b7e1f005cf049&scope.project.id=2e3a9af772214953842e9cc63a52ae3a&role.id=b43d5a8247e244869c2c70f986f42e8a => generated 611 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:52.226721 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffc902b-c774-416b-a2d3-91b55afb0e52 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:52.227204 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffc902b-c774-416b-a2d3-91b55afb0e52 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:52.227204 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffc902b-c774-416b-a2d3-91b55afb0e52 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:52.229336 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 221/456] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:44:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:44:52.231015 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6dbd6b-240e-435d-9217-11e2b3318cfd None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:52.231449 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6dbd6b-240e-435d-9217-11e2b3318cfd None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:52.231616 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6dbd6b-240e-435d-9217-11e2b3318cfd None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:52.233213 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 236/457] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:44:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:44:52.238472 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3c6b1b-e638-421e-ab7f-007ea30d08af None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:52.238757 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3c6b1b-e638-421e-ab7f-007ea30d08af None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:52.239045 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3c6b1b-e638-421e-ab7f-007ea30d08af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:52.248609 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c76c956-561e-4024-b47a-053c3f9ac9e1 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:52.253971 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c76c956-561e-4024-b47a-053c3f9ac9e1 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:52.253971 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c76c956-561e-4024-b47a-053c3f9ac9e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:52.265475 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-2f3c6b1b-e638-421e-ab7f-007ea30d08af None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:44:52.266788 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2f3c6b1b-e638-421e-ab7f-007ea30d08af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:44:52.285133 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-8c76c956-561e-4024-b47a-053c3f9ac9e1 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:44:52.287439 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-8c76c956-561e-4024-b47a-053c3f9ac9e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:44:52.372858 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 222/458] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:44:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 10 12:44:52.374399 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbea535-d763-44ba-aae0-6d8343a60bfa None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:52.374695 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbea535-d763-44ba-aae0-6d8343a60bfa None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:52.374978 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbea535-d763-44ba-aae0-6d8343a60bfa None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:52.376280 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 223/459] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:44:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:44:52.381418 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 237/460] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:44:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 10 12:44:52.389612 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32ff139-cd5f-40ec-b512-69d959af56ae None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:52.389911 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32ff139-cd5f-40ec-b512-69d959af56ae None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:52.390199 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32ff139-cd5f-40ec-b512-69d959af56ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:52.403393 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c76c956-561e-4024-b47a-053c3f9ac9e1 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:52.420476 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:52.420904 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-d32ff139-cd5f-40ec-b512-69d959af56ae None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:44:52.422103 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d32ff139-cd5f-40ec-b512-69d959af56ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:44:52.482539 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:52.495203 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 224/461] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:44:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 10 12:44:52.504453 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32ff139-cd5f-40ec-b512-69d959af56ae None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:52.526947 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c76c956-561e-4024-b47a-053c3f9ac9e1 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:52.576600 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346e834f-6487-42ad-9583-11684ffe1ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:52.577537 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346e834f-6487-42ad-9583-11684ffe1ff3 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:52.577955 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346e834f-6487-42ad-9583-11684ffe1ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:52.578462 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346e834f-6487-42ad-9583-11684ffe1ff3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:52.589420 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346e834f-6487-42ad-9583-11684ffe1ff3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:52.591137 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346e834f-6487-42ad-9583-11684ffe1ff3 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:52.612937 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-346e834f-6487-42ad-9583-11684ffe1ff3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:44:52.612937 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:44:52.612937 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:44:52.612937 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:44:52.612937 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:44:52.612937 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:44:52.612937 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:44:52.612937 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:44:52.612937 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:44:52.612937 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:44:52.612937 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:44:52.612937 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:44:52.612937 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 10 12:44:52.612937 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 10 12:44:52.612937 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 10 12:44:52.613888 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 10 12:44:52.613888 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:44:52.613888 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:44:52.613888 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:44:52.613888 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:44:52.613888 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:44:52.613888 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:44:52.613888 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:44:52.613888 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:44:52.613888 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:44:52.613888 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:44:52.613888 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:44:52.613888 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:44:52.613888 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:44:52.613888 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:44:52.613888 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:44:52.613888 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:44:52.615150 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 10 12:44:52.615150 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 10 12:44:52.615150 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:44:52.615150 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:44:52.615150 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 238/462] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:44:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 10 12:44:52.615427 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346e834f-6487-42ad-9583-11684ffe1ff3 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:52.625186 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32ff139-cd5f-40ec-b512-69d959af56ae None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:52.659991 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25bfe36-ae09-40c7-8fdf-25d4947b7002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:52.660795 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25bfe36-ae09-40c7-8fdf-25d4947b7002 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:52.661082 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25bfe36-ae09-40c7-8fdf-25d4947b7002 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:52.661353 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25bfe36-ae09-40c7-8fdf-25d4947b7002 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:52.672407 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25bfe36-ae09-40c7-8fdf-25d4947b7002 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:52.673880 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25bfe36-ae09-40c7-8fdf-25d4947b7002 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:52.686578 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:52.692309 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-a25bfe36-ae09-40c7-8fdf-25d4947b7002 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:44:52.692309 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:44:52.692309 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:44:52.692309 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:44:52.692309 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:44:52.692309 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:44:52.692309 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:44:52.692309 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:44:52.692309 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:44:52.692309 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:44:52.692309 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:44:52.692309 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:44:52.692309 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 10 12:44:52.692309 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 10 12:44:52.692309 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 10 12:44:52.693357 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 10 12:44:52.693357 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:44:52.693357 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:44:52.693357 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:44:52.693357 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:44:52.693357 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:44:52.693357 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:44:52.693357 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:44:52.693357 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:44:52.693357 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:44:52.693357 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:44:52.693357 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:44:52.693357 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:44:52.693357 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:44:52.693357 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:44:52.693357 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:44:52.693357 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:44:52.694344 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 10 12:44:52.694344 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 10 12:44:52.694344 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:44:52.694344 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:44:52.694344 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 225/463] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:44:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:44:52.695020 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c422b2d5-2584-4ead-8e07-fc90cd687ea9 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:52.695311 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c422b2d5-2584-4ead-8e07-fc90cd687ea9 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:52.695587 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c422b2d5-2584-4ead-8e07-fc90cd687ea9 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:52.696897 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 226/464] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:44:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:44:52.697706 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c422b2d5-2584-4ead-8e07-fc90cd687ea9 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:52.736033 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346e834f-6487-42ad-9583-11684ffe1ff3 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:52.748440 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:52.772523 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43117d7-61d7-49c0-b535-e2399c430d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:52.773314 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43117d7-61d7-49c0-b535-e2399c430d0d None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:52.773602 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43117d7-61d7-49c0-b535-e2399c430d0d None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:52.773878 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43117d7-61d7-49c0-b535-e2399c430d0d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:52.785405 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43117d7-61d7-49c0-b535-e2399c430d0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:52.787316 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43117d7-61d7-49c0-b535-e2399c430d0d None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:52.805211 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-f43117d7-61d7-49c0-b535-e2399c430d0d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:44:52.805211 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:44:52.805211 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:44:52.805211 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:44:52.805211 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:44:52.805211 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:44:52.805211 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:44:52.805211 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:44:52.805211 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:44:52.805211 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:44:52.805211 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:44:52.805211 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:44:52.805211 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 10 12:44:52.805211 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 10 12:44:52.805211 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 10 12:44:52.806721 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 10 12:44:52.806721 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:44:52.806721 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:44:52.806721 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:44:52.806721 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:44:52.806721 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:44:52.806721 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:44:52.806721 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:44:52.806721 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:44:52.806721 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:44:52.806721 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:44:52.806721 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:44:52.806721 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:44:52.806721 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:44:52.806721 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:44:52.806721 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:44:52.806721 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:44:52.807602 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 10 12:44:52.807602 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 10 12:44:52.807602 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:44:52.807602 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:44:52.807602 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 239/465] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:44:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:44:52.807899 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43117d7-61d7-49c0-b535-e2399c430d0d None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:52.812935 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c422b2d5-2584-4ead-8e07-fc90cd687ea9 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:52.851520 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cc6022-a652-4f34-96b2-fa2f22120b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:52.852223 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cc6022-a652-4f34-96b2-fa2f22120b12 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:52.853083 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cc6022-a652-4f34-96b2-fa2f22120b12 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:52.853368 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cc6022-a652-4f34-96b2-fa2f22120b12 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:52.854102 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cc6022-a652-4f34-96b2-fa2f22120b12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:52.854409 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cc6022-a652-4f34-96b2-fa2f22120b12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:52.855795 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cc6022-a652-4f34-96b2-fa2f22120b12 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:52.867884 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 227/466] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Jun 10 12:44:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:52.869250 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e93e2a4-b092-4664-99b1-e320803e78b6 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:52.869540 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e93e2a4-b092-4664-99b1-e320803e78b6 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:52.869822 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e93e2a4-b092-4664-99b1-e320803e78b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:52.886758 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-0e93e2a4-b092-4664-99b1-e320803e78b6 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:44:52.887984 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0e93e2a4-b092-4664-99b1-e320803e78b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:44:52.929292 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43117d7-61d7-49c0-b535-e2399c430d0d None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:52.944272 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:52.957777 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 228/467] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:44:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 10 12:44:52.960956 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e93e2a4-b092-4664-99b1-e320803e78b6 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:52.967144 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fe4f5d-734a-4ad7-ba9d-30c64ff72a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:52.967144 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fe4f5d-734a-4ad7-ba9d-30c64ff72a45 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:52.967144 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fe4f5d-734a-4ad7-ba9d-30c64ff72a45 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:52.967144 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fe4f5d-734a-4ad7-ba9d-30c64ff72a45 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:52.967461 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fe4f5d-734a-4ad7-ba9d-30c64ff72a45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:52.967461 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fe4f5d-734a-4ad7-ba9d-30c64ff72a45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:52.968251 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fe4f5d-734a-4ad7-ba9d-30c64ff72a45 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:52.985443 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 240/468] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Jun 10 12:44:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:52.985443 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fe4f5d-734a-4ad7-ba9d-30c64ff72a45 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:53.003212 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:53.075500 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e93e2a4-b092-4664-99b1-e320803e78b6 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:53.083939 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fe4f5d-734a-4ad7-ba9d-30c64ff72a45 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:53.112635 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55247de-9340-405f-a248-68e0eaa9984e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:53.113311 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55247de-9340-405f-a248-68e0eaa9984e None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:53.113592 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55247de-9340-405f-a248-68e0eaa9984e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:53.113871 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55247de-9340-405f-a248-68e0eaa9984e None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:53.114597 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55247de-9340-405f-a248-68e0eaa9984e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:53.114905 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55247de-9340-405f-a248-68e0eaa9984e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:53.116317 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55247de-9340-405f-a248-68e0eaa9984e None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:53.121436 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091b3647-9a81-4a15-8f3b-9f7ffa03da62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:53.122109 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091b3647-9a81-4a15-8f3b-9f7ffa03da62 None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:53.122402 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091b3647-9a81-4a15-8f3b-9f7ffa03da62 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:53.122679 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091b3647-9a81-4a15-8f3b-9f7ffa03da62 None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:53.123387 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091b3647-9a81-4a15-8f3b-9f7ffa03da62 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:53.124856 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091b3647-9a81-4a15-8f3b-9f7ffa03da62 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:53.128472 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 229/469] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Jun 10 12:44:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:53.133333 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55247de-9340-405f-a248-68e0eaa9984e None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:53.225322 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 241/470] 10.222.0.27 () {60 vars in 1132 bytes} [Fri Jun 10 12:44:52 2022] POST /identity/v3/users => generated 248 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 12:44:53.226285 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091b3647-9a81-4a15-8f3b-9f7ffa03da62 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:53.236617 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:53.253097 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55247de-9340-405f-a248-68e0eaa9984e None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:53.275089 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:53.293718 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090a81bf-c12f-40fb-a3e3-d249bac14b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:53.295516 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090a81bf-c12f-40fb-a3e3-d249bac14b7d None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:53.295516 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090a81bf-c12f-40fb-a3e3-d249bac14b7d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:53.295516 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090a81bf-c12f-40fb-a3e3-d249bac14b7d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:53.306797 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090a81bf-c12f-40fb-a3e3-d249bac14b7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:53.308321 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090a81bf-c12f-40fb-a3e3-d249bac14b7d None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:53.328172 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-090a81bf-c12f-40fb-a3e3-d249bac14b7d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:44:53.328172 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:44:53.328172 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:44:53.328172 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:44:53.328172 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:44:53.328172 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:44:53.328172 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:44:53.328172 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:44:53.328172 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:44:53.328172 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:44:53.328172 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:44:53.328172 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:44:53.328172 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 10 12:44:53.328172 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 10 12:44:53.328172 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 10 12:44:53.329121 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 10 12:44:53.329121 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:44:53.329121 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:44:53.329121 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:44:53.329121 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:44:53.329121 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:44:53.329121 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:44:53.329121 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:44:53.329121 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:44:53.329121 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:44:53.329121 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:44:53.329121 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:44:53.329121 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:44:53.329121 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:44:53.329121 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:44:53.329121 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:44:53.329121 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:44:53.329973 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 10 12:44:53.329973 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 10 12:44:53.329973 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:44:53.329973 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:44:53.329973 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 230/471] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:44:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:44:53.330243 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090a81bf-c12f-40fb-a3e3-d249bac14b7d None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:53.362689 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091b3647-9a81-4a15-8f3b-9f7ffa03da62 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:53.402039 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66e6f89-833e-4a72-8f23-3836545e78aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:53.402719 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66e6f89-833e-4a72-8f23-3836545e78aa None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:53.403424 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66e6f89-833e-4a72-8f23-3836545e78aa None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:53.403424 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66e6f89-833e-4a72-8f23-3836545e78aa None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:53.404013 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66e6f89-833e-4a72-8f23-3836545e78aa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:53.405456 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66e6f89-833e-4a72-8f23-3836545e78aa None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:53.443460 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090a81bf-c12f-40fb-a3e3-d249bac14b7d None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:53.471337 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 242/472] 10.222.0.27 () {60 vars in 1132 bytes} [Fri Jun 10 12:44:53 2022] POST /identity/v3/users => generated 248 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 12:44:53.472848 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d059e740-ead3-4200-ac06-65023b57ab12 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:53.473139 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d059e740-ead3-4200-ac06-65023b57ab12 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:53.473424 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d059e740-ead3-4200-ac06-65023b57ab12 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:53.474870 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 243/473] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:44:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:44:53.475392 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d059e740-ead3-4200-ac06-65023b57ab12 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:53.488309 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44591f80-deac-4ef3-8461-7d081bc09ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:53.489012 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44591f80-deac-4ef3-8461-7d081bc09ad6 None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:53.489296 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44591f80-deac-4ef3-8461-7d081bc09ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:53.489576 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44591f80-deac-4ef3-8461-7d081bc09ad6 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:53.490369 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44591f80-deac-4ef3-8461-7d081bc09ad6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:53.491939 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44591f80-deac-4ef3-8461-7d081bc09ad6 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:53.575644 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:53.586737 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 231/474] 10.222.0.27 () {60 vars in 1132 bytes} [Fri Jun 10 12:44:53 2022] POST /identity/v3/users => generated 246 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 12:44:53.588830 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86da54a4-a89b-41c5-bae3-40813035de01 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:53.589125 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86da54a4-a89b-41c5-bae3-40813035de01 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:53.589410 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86da54a4-a89b-41c5-bae3-40813035de01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:53.593497 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:53.618942 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d059e740-ead3-4200-ac06-65023b57ab12 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:53.624210 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-86da54a4-a89b-41c5-bae3-40813035de01 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:44:53.625405 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-86da54a4-a89b-41c5-bae3-40813035de01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:44:53.685524 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebb7f91-0246-456e-ab15-9ea278300a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:53.686687 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebb7f91-0246-456e-ab15-9ea278300a39 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:53.687991 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebb7f91-0246-456e-ab15-9ea278300a39 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:53.689419 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebb7f91-0246-456e-ab15-9ea278300a39 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:53.690355 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebb7f91-0246-456e-ab15-9ea278300a39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:53.690815 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebb7f91-0246-456e-ab15-9ea278300a39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:53.692524 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebb7f91-0246-456e-ab15-9ea278300a39 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:53.709498 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 244/475] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Jun 10 12:44:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:53.718352 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebb7f91-0246-456e-ab15-9ea278300a39 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:53.751744 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 232/476] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:44:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 10 12:44:53.777533 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86da54a4-a89b-41c5-bae3-40813035de01 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:53.845351 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebb7f91-0246-456e-ab15-9ea278300a39 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:53.886542 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a215243-9f31-42ec-a808-ec55c9bb7413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:53.887776 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a215243-9f31-42ec-a808-ec55c9bb7413 None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:53.888188 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a215243-9f31-42ec-a808-ec55c9bb7413 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:53.888563 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a215243-9f31-42ec-a808-ec55c9bb7413 None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:53.890089 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a215243-9f31-42ec-a808-ec55c9bb7413 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:53.891641 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a215243-9f31-42ec-a808-ec55c9bb7413 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:53.892466 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86da54a4-a89b-41c5-bae3-40813035de01 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:53.933270 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:53.940420 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e202ef80-245d-4a6d-bf39-47dade030cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:53.941128 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e202ef80-245d-4a6d-bf39-47dade030cf5 None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:53.941419 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e202ef80-245d-4a6d-bf39-47dade030cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:53.941695 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e202ef80-245d-4a6d-bf39-47dade030cf5 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:53.942483 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e202ef80-245d-4a6d-bf39-47dade030cf5 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:53.944773 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e202ef80-245d-4a6d-bf39-47dade030cf5 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:53.976948 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 245/477] 10.222.0.27 () {60 vars in 1132 bytes} [Fri Jun 10 12:44:53 2022] POST /identity/v3/users => generated 249 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 12:44:54.019461 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.sql.core [None req-e202ef80-245d-4a6d-bf39-47dade030cf5 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 10 12:44:54.019461 dv-cinder-master-3-smb devstack@keystone.service[80237]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 10 12:44:54.019461 dv-cinder-master-3-smb devstack@keystone.service[80237]: [parameters: {'id': 'e6133e247438493bb46fb36d7579d23e', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 10 12:44:54.019461 dv-cinder-master-3-smb devstack@keystone.service[80237]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80237) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 10 12:44:54.041718 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-e202ef80-245d-4a6d-bf39-47dade030cf5 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 10 12:44:54.041718 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:44:54.041718 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 10 12:44:54.041718 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 10 12:44:54.041718 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 10 12:44:54.041718 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 10 12:44:54.041718 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 10 12:44:54.041718 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application result = self._query(query) Jun 10 12:44:54.041718 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 10 12:44:54.041718 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application conn.query(q) Jun 10 12:44:54.041718 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 10 12:44:54.041718 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 10 12:44:54.041718 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 10 12:44:54.041718 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application result.read() Jun 10 12:44:54.041718 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 10 12:44:54.042672 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 10 12:44:54.042672 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 10 12:44:54.042672 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 10 12:44:54.042672 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 10 12:44:54.042672 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 10 12:44:54.042672 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 10 12:44:54.042672 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 10 12:44:54.042672 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 10 12:44:54.042672 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:44:54.042672 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 10 12:44:54.042672 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:44:54.042672 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:44:54.042672 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 10 12:44:54.042672 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 10 12:44:54.042672 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 10 12:44:54.042672 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return ref.to_dict() Jun 10 12:44:54.042672 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 10 12:44:54.042672 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application next(self.gen) Jun 10 12:44:54.043623 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 10 12:44:54.043623 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application yield resource Jun 10 12:44:54.043623 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 10 12:44:54.043623 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application next(self.gen) Jun 10 12:44:54.043623 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 10 12:44:54.043623 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application self.session.rollback() Jun 10 12:44:54.043623 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 10 12:44:54.043623 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application self.force_reraise() Jun 10 12:44:54.043623 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 10 12:44:54.043623 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise self.value Jun 10 12:44:54.043623 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 10 12:44:54.043623 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 10 12:44:54.043623 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 10 12:44:54.043623 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application session.commit() Jun 10 12:44:54.043623 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 10 12:44:54.043623 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 10 12:44:54.043623 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 10 12:44:54.044555 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._prepare_impl() Jun 10 12:44:54.044555 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 10 12:44:54.044555 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application self.session.flush() Jun 10 12:44:54.044555 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 10 12:44:54.044555 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._flush(objects) Jun 10 12:44:54.044555 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 10 12:44:54.044555 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 10 12:44:54.044555 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 10 12:44:54.044555 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application compat.raise_( Jun 10 12:44:54.044555 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 10 12:44:54.044555 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception Jun 10 12:44:54.044555 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 10 12:44:54.044555 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application flush_context.execute() Jun 10 12:44:54.044555 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 10 12:44:54.044555 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rec.execute(self) Jun 10 12:44:54.044555 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 10 12:44:54.044555 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 10 12:44:54.045464 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 10 12:44:54.045464 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 10 12:44:54.045464 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 10 12:44:54.045464 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 10 12:44:54.045464 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 10 12:44:54.045464 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 10 12:44:54.045464 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 10 12:44:54.045464 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 10 12:44:54.045464 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 10 12:44:54.045464 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 10 12:44:54.045464 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 10 12:44:54.045464 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 10 12:44:54.045464 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 10 12:44:54.045464 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 10 12:44:54.045464 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 10 12:44:54.045464 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception Jun 10 12:44:54.046301 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 10 12:44:54.046301 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 10 12:44:54.046301 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 10 12:44:54.046301 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 10 12:44:54.046301 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 10 12:44:54.046301 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application result = self._query(query) Jun 10 12:44:54.046301 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 10 12:44:54.046301 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application conn.query(q) Jun 10 12:44:54.046301 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 10 12:44:54.046301 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 10 12:44:54.046301 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 10 12:44:54.046301 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application result.read() Jun 10 12:44:54.046301 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 10 12:44:54.046301 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 10 12:44:54.046301 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 10 12:44:54.046301 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 10 12:44:54.047100 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 10 12:44:54.047100 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 10 12:44:54.047100 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 10 12:44:54.047100 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 10 12:44:54.047100 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 10 12:44:54.047100 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 10 12:44:54.047100 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [parameters: {'id': 'e6133e247438493bb46fb36d7579d23e', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 10 12:44:54.047100 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 10 12:44:54.047100 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:44:54.047100 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 10 12:44:54.047100 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:44:54.047100 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:44:54.047100 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:44:54.047100 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:44:54.047100 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:44:54.047100 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:44:54.048140 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:44:54.048140 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:44:54.048140 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:44:54.048140 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:44:54.048140 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:44:54.048140 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:44:54.048140 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 10 12:44:54.048140 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 10 12:44:54.048140 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:44:54.048140 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:44:54.048140 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 10 12:44:54.048140 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 10 12:44:54.048140 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 10 12:44:54.048140 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 10 12:44:54.048140 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 10 12:44:54.048140 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:44:54.049034 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 233/478] 10.222.0.27 () {60 vars in 1131 bytes} [Fri Jun 10 12:44:53 2022] POST /identity/v3/roles => generated 144 bytes in 266 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 10 12:44:54.054838 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a215243-9f31-42ec-a808-ec55c9bb7413 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:54.062091 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:54.196178 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a215243-9f31-42ec-a808-ec55c9bb7413 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:54.231779 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f573336-e6e1-4793-ba36-f87eaa256997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:54.232681 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f573336-e6e1-4793-ba36-f87eaa256997 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:54.232994 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f573336-e6e1-4793-ba36-f87eaa256997 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:54.233285 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f573336-e6e1-4793-ba36-f87eaa256997 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:54.245402 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f573336-e6e1-4793-ba36-f87eaa256997 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:44:54.254145 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-9f573336-e6e1-4793-ba36-f87eaa256997 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 10 12:44:54.254145 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:44:54.254145 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:44:54.254145 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:44:54.254145 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:44:54.254145 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:44:54.254145 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:44:54.254145 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:44:54.254145 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:44:54.254145 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:44:54.254145 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:44:54.254145 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:44:54.254145 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:44:54.254145 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:44:54.254145 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:44:54.254145 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:44:54.255071 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:44:54.255071 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:44:54.255071 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:44:54.255071 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:44:54.255071 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:44:54.255071 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:44:54.255071 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:44:54.255071 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:44:54.255071 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:44:54.255071 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:44:54.255071 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:44:54.255071 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:44:54.255071 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:44:54.255071 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:44:54.255071 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:44:54.255071 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:44:54.255912 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:44:54.255912 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:44:54.255912 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:44:54.255912 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:44:54.255912 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:44:54.255912 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:44:54.255912 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:44:54.255912 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:44:54.255912 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:44:54.255912 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:44:54.255912 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 10 12:44:54.255912 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:44:54.258304 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 246/479] 10.222.0.27 () {56 vars in 1114 bytes} [Fri Jun 10 12:44:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:44:54.264073 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e202ef80-245d-4a6d-bf39-47dade030cf5 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:54.377902 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e202ef80-245d-4a6d-bf39-47dade030cf5 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:54.416547 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd5acef-138e-4857-8721-c14b182db477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:54.417477 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd5acef-138e-4857-8721-c14b182db477 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:54.417911 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd5acef-138e-4857-8721-c14b182db477 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:54.418335 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd5acef-138e-4857-8721-c14b182db477 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:54.419251 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd5acef-138e-4857-8721-c14b182db477 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:54.419704 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd5acef-138e-4857-8721-c14b182db477 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:54.421241 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd5acef-138e-4857-8721-c14b182db477 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:54.430028 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 234/480] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Jun 10 12:44:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:55.688917 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912bb35f-a730-40a2-97fe-489c12b1d364 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:55.689464 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912bb35f-a730-40a2-97fe-489c12b1d364 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:55.689464 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912bb35f-a730-40a2-97fe-489c12b1d364 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:55.691058 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 247/481] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:44:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:44:55.696469 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36006da5-abce-47be-bf1c-eea6970463c3 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:55.696892 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36006da5-abce-47be-bf1c-eea6970463c3 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:55.697089 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36006da5-abce-47be-bf1c-eea6970463c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:55.718327 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-36006da5-abce-47be-bf1c-eea6970463c3 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:44:55.719500 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-36006da5-abce-47be-bf1c-eea6970463c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:44:55.785235 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:55.803645 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 235/482] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:44:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 10 12:44:55.839412 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912bb35f-a730-40a2-97fe-489c12b1d364 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:55.925053 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:55.977193 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912bb35f-a730-40a2-97fe-489c12b1d364 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:56.036558 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba385eb8-c52e-4646-9808-8c5eea233017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:56.037612 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba385eb8-c52e-4646-9808-8c5eea233017 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:56.037893 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba385eb8-c52e-4646-9808-8c5eea233017 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:56.038164 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba385eb8-c52e-4646-9808-8c5eea233017 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:56.054536 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba385eb8-c52e-4646-9808-8c5eea233017 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:44:56.062395 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-ba385eb8-c52e-4646-9808-8c5eea233017 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 10 12:44:56.062395 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:44:56.062395 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:44:56.062395 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:44:56.062395 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:44:56.062395 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:44:56.062395 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:44:56.062395 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:44:56.062395 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:44:56.062395 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:44:56.062395 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:44:56.062395 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:44:56.062395 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:44:56.062395 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:44:56.062395 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:44:56.062395 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:44:56.063476 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:44:56.063476 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:44:56.063476 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:44:56.063476 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:44:56.063476 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:44:56.063476 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:44:56.063476 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:44:56.063476 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:44:56.063476 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:44:56.063476 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:44:56.063476 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:44:56.063476 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:44:56.063476 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:44:56.063476 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:44:56.063476 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:44:56.063476 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:44:56.064320 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:44:56.064320 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:44:56.064320 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:44:56.064320 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:44:56.064320 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:44:56.064320 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:44:56.064320 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:44:56.064320 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:44:56.064320 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:44:56.064320 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:44:56.064320 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 10 12:44:56.064320 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:44:56.064320 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 248/483] 10.222.0.27 () {56 vars in 1102 bytes} [Fri Jun 10 12:44:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:44:56.069649 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36006da5-abce-47be-bf1c-eea6970463c3 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:56.151672 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a781cacf-1afd-4ac3-93a2-62b95b8265a0 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:56.152563 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a781cacf-1afd-4ac3-93a2-62b95b8265a0 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:56.152920 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a781cacf-1afd-4ac3-93a2-62b95b8265a0 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:56.154694 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 249/484] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:44:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 10 12:44:56.160857 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f263b5-5d52-493a-b99a-0e4a16bf12d0 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:56.161143 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f263b5-5d52-493a-b99a-0e4a16bf12d0 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:56.164431 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f263b5-5d52-493a-b99a-0e4a16bf12d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:56.229230 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-16f263b5-5d52-493a-b99a-0e4a16bf12d0 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:44:56.234505 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-16f263b5-5d52-493a-b99a-0e4a16bf12d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:44:56.237924 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36006da5-abce-47be-bf1c-eea6970463c3 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:56.278077 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04db1f35-04ed-4375-a360-0ae87ee957d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:56.278746 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04db1f35-04ed-4375-a360-0ae87ee957d8 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:56.279031 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04db1f35-04ed-4375-a360-0ae87ee957d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:56.279312 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04db1f35-04ed-4375-a360-0ae87ee957d8 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:56.280090 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04db1f35-04ed-4375-a360-0ae87ee957d8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:56.280397 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04db1f35-04ed-4375-a360-0ae87ee957d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:56.281774 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04db1f35-04ed-4375-a360-0ae87ee957d8 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:56.300090 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 236/485] 10.222.0.27 () {56 vars in 1103 bytes} [Fri Jun 10 12:44:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:56.302364 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04db1f35-04ed-4375-a360-0ae87ee957d8 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:56.384212 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 250/486] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:44:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 10 12:44:56.385683 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ce7903-414a-42bc-a442-0954a408e99c None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:56.385974 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ce7903-414a-42bc-a442-0954a408e99c None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:56.386253 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ce7903-414a-42bc-a442-0954a408e99c None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:56.387616 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:56.388414 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 251/487] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:44:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:44:56.393669 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633098e9-f83e-4553-9a58-f966fbe27fbc None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:56.393962 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633098e9-f83e-4553-9a58-f966fbe27fbc None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:56.394245 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633098e9-f83e-4553-9a58-f966fbe27fbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:56.411529 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-633098e9-f83e-4553-9a58-f966fbe27fbc None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:44:56.412973 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-633098e9-f83e-4553-9a58-f966fbe27fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:44:56.455765 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04db1f35-04ed-4375-a360-0ae87ee957d8 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:56.468130 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:56.490339 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f0d55e-c157-4aab-b74f-98a529eea91e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:56.491070 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f0d55e-c157-4aab-b74f-98a529eea91e None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:56.491565 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f0d55e-c157-4aab-b74f-98a529eea91e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:56.491672 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f0d55e-c157-4aab-b74f-98a529eea91e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:56.495962 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 252/488] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:44:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 10 12:44:56.496859 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633098e9-f83e-4553-9a58-f966fbe27fbc None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:56.503017 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f0d55e-c157-4aab-b74f-98a529eea91e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:56.504500 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f0d55e-c157-4aab-b74f-98a529eea91e None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:56.522423 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-16f0d55e-c157-4aab-b74f-98a529eea91e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:44:56.522423 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:44:56.522423 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:44:56.522423 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:44:56.522423 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:44:56.522423 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:44:56.522423 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:44:56.522423 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:44:56.522423 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:44:56.522423 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:44:56.522423 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:44:56.522423 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:44:56.522423 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 10 12:44:56.522423 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 10 12:44:56.522423 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 10 12:44:56.523318 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 10 12:44:56.523318 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:44:56.523318 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:44:56.523318 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:44:56.523318 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:44:56.523318 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:44:56.523318 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:44:56.523318 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:44:56.523318 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:44:56.523318 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:44:56.523318 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:44:56.523318 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:44:56.523318 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:44:56.523318 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:44:56.523318 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:44:56.523318 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:44:56.523318 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:44:56.524786 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 10 12:44:56.524786 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 10 12:44:56.524786 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:44:56.524786 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:44:56.527970 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 237/489] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:44:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 10 12:44:56.529824 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f0d55e-c157-4aab-b74f-98a529eea91e None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:56.620108 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633098e9-f83e-4553-9a58-f966fbe27fbc None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:56.636556 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:56.640317 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f0d55e-c157-4aab-b74f-98a529eea91e None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:56.655725 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072470e8-2931-4c57-95b8-850f0a398779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:56.656630 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072470e8-2931-4c57-95b8-850f0a398779 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:56.656911 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072470e8-2931-4c57-95b8-850f0a398779 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:56.657188 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072470e8-2931-4c57-95b8-850f0a398779 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:56.668184 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072470e8-2931-4c57-95b8-850f0a398779 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:44:56.676123 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-072470e8-2931-4c57-95b8-850f0a398779 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 10 12:44:56.676123 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:44:56.676123 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:44:56.676123 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:44:56.676123 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:44:56.676123 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:44:56.676123 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:44:56.676123 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:44:56.676123 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:44:56.676123 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:44:56.676123 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:44:56.676123 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:44:56.676123 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:44:56.676123 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:44:56.676123 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:44:56.676123 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:44:56.677019 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:44:56.677019 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:44:56.677019 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:44:56.677019 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:44:56.677019 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:44:56.677019 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:44:56.677019 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:44:56.677019 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:44:56.677019 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:44:56.677019 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:44:56.677019 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:44:56.677019 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:44:56.677019 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:44:56.677019 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:44:56.677019 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:44:56.677019 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:44:56.677828 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:44:56.677828 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:44:56.677828 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:44:56.677828 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:44:56.677828 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:44:56.677828 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:44:56.677828 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:44:56.677828 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:44:56.677828 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:44:56.677828 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:44:56.677828 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 10 12:44:56.677828 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:44:56.679306 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94fbcc3-b4d3-486a-86be-a664568ca570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:56.680822 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94fbcc3-b4d3-486a-86be-a664568ca570 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:56.682359 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94fbcc3-b4d3-486a-86be-a664568ca570 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:56.682710 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 253/490] 10.222.0.27 () {56 vars in 1102 bytes} [Fri Jun 10 12:44:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:44:56.683064 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94fbcc3-b4d3-486a-86be-a664568ca570 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:56.683549 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072470e8-2931-4c57-95b8-850f0a398779 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:56.684621 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94fbcc3-b4d3-486a-86be-a664568ca570 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:56.685135 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94fbcc3-b4d3-486a-86be-a664568ca570 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:56.686787 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94fbcc3-b4d3-486a-86be-a664568ca570 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:56.701070 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 238/491] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Jun 10 12:44:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:56.707003 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94fbcc3-b4d3-486a-86be-a664568ca570 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:56.731288 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:56.785741 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072470e8-2931-4c57-95b8-850f0a398779 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:56.818241 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94fbcc3-b4d3-486a-86be-a664568ca570 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:56.824428 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7339e57f-2505-480f-9925-17b2ac746379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:56.825227 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7339e57f-2505-480f-9925-17b2ac746379 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:56.825510 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7339e57f-2505-480f-9925-17b2ac746379 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:56.825786 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7339e57f-2505-480f-9925-17b2ac746379 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:56.836361 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7339e57f-2505-480f-9925-17b2ac746379 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:44:56.844064 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-7339e57f-2505-480f-9925-17b2ac746379 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 10 12:44:56.844064 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:44:56.844064 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:44:56.844064 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:44:56.844064 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:44:56.844064 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:44:56.844064 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:44:56.844064 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:44:56.844064 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:44:56.844064 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:44:56.844064 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:44:56.844064 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:44:56.844064 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:44:56.844064 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:44:56.844064 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:44:56.844064 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:44:56.844976 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:44:56.844976 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:44:56.844976 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:44:56.844976 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:44:56.844976 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:44:56.844976 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:44:56.844976 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:44:56.844976 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:44:56.844976 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:44:56.844976 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:44:56.844976 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:44:56.844976 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:44:56.844976 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:44:56.844976 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:44:56.844976 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:44:56.844976 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:44:56.845798 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:44:56.845798 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:44:56.845798 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:44:56.845798 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:44:56.845798 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:44:56.845798 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:44:56.845798 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:44:56.845798 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:44:56.845798 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:44:56.845798 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:44:56.845798 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 10 12:44:56.845798 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:44:56.853014 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 254/492] 10.222.0.27 () {56 vars in 1102 bytes} [Fri Jun 10 12:44:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:44:56.853875 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7339e57f-2505-480f-9925-17b2ac746379 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:56.865258 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9933111f-fc5a-4878-81a8-a45e9dfbb852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:56.866389 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9933111f-fc5a-4878-81a8-a45e9dfbb852 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:56.866671 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9933111f-fc5a-4878-81a8-a45e9dfbb852 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:56.866947 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9933111f-fc5a-4878-81a8-a45e9dfbb852 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:56.868685 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9933111f-fc5a-4878-81a8-a45e9dfbb852 None admin] RBAC: Adding query filter params (name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:56.869152 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9933111f-fc5a-4878-81a8-a45e9dfbb852 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:56.870512 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9933111f-fc5a-4878-81a8-a45e9dfbb852 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:56.878913 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 239/493] 10.222.0.27 () {56 vars in 1103 bytes} [Fri Jun 10 12:44:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:56.879688 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9933111f-fc5a-4878-81a8-a45e9dfbb852 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:56.906747 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:56.967849 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7339e57f-2505-480f-9925-17b2ac746379 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:56.969546 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:56.982796 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9933111f-fc5a-4878-81a8-a45e9dfbb852 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:57.003782 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c9cac7-f5c1-4544-8c6a-639f995a36d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:57.004600 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c9cac7-f5c1-4544-8c6a-639f995a36d1 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:57.004886 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c9cac7-f5c1-4544-8c6a-639f995a36d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:57.005158 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c9cac7-f5c1-4544-8c6a-639f995a36d1 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:57.012244 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c9cac7-f5c1-4544-8c6a-639f995a36d1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:57.013698 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c9cac7-f5c1-4544-8c6a-639f995a36d1 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:57.019722 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cde4c11-b12c-4b1b-96a3-660062b36d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:57.020605 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cde4c11-b12c-4b1b-96a3-660062b36d16 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:57.021040 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cde4c11-b12c-4b1b-96a3-660062b36d16 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:57.021467 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cde4c11-b12c-4b1b-96a3-660062b36d16 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:57.022363 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cde4c11-b12c-4b1b-96a3-660062b36d16 None admin] RBAC: Adding query filter params (name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:57.022818 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cde4c11-b12c-4b1b-96a3-660062b36d16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:57.024483 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cde4c11-b12c-4b1b-96a3-660062b36d16 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:57.029126 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-65c9cac7-f5c1-4544-8c6a-639f995a36d1 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 10 12:44:57.029126 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:44:57.029126 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:44:57.029126 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:44:57.029126 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:44:57.029126 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:44:57.029126 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:44:57.029126 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:44:57.029126 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:44:57.029126 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:44:57.029126 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:44:57.029126 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:44:57.029126 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 10 12:44:57.029126 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 10 12:44:57.029126 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 10 12:44:57.029126 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 10 12:44:57.030221 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:44:57.030221 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:44:57.030221 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 10 12:44:57.030221 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 10 12:44:57.030221 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 10 12:44:57.030221 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 10 12:44:57.030221 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:44:57.030221 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:44:57.030221 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:44:57.030221 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:44:57.030221 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:44:57.030221 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:44:57.030221 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:44:57.030221 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:44:57.030221 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:44:57.030221 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:44:57.030221 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:44:57.030221 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:44:57.031157 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:44:57.031157 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:44:57.031157 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 10 12:44:57.031157 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 10 12:44:57.031157 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 10 12:44:57.031157 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 10 12:44:57.031157 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 10 12:44:57.031157 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 10 12:44:57.031157 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 10 12:44:57.031157 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:44:57.031157 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 255/494] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Jun 10 12:44:56 2022] GET /identity/v3/users/glance => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:44:57.031727 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24802e1-d56c-40d6-a21e-47e9943d13a1 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:57.031829 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24802e1-d56c-40d6-a21e-47e9943d13a1 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:57.032131 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24802e1-d56c-40d6-a21e-47e9943d13a1 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:57.035967 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 256/495] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:44:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:44:57.035967 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24802e1-d56c-40d6-a21e-47e9943d13a1 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:57.045968 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 240/496] 10.222.0.27 () {56 vars in 1103 bytes} [Fri Jun 10 12:44:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:57.045968 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cde4c11-b12c-4b1b-96a3-660062b36d16 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:57.134291 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:57.149718 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cde4c11-b12c-4b1b-96a3-660062b36d16 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:57.150085 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24802e1-d56c-40d6-a21e-47e9943d13a1 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:57.183639 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c3a1b0-73f5-4d4d-9927-4b77ad4f896c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:57.184309 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf2227d-edf5-410b-a35b-d07eae77a1ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:57.184432 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c3a1b0-73f5-4d4d-9927-4b77ad4f896c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:57.184609 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c3a1b0-73f5-4d4d-9927-4b77ad4f896c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:57.184889 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c3a1b0-73f5-4d4d-9927-4b77ad4f896c None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:57.185094 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf2227d-edf5-410b-a35b-d07eae77a1ac None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:57.185378 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf2227d-edf5-410b-a35b-d07eae77a1ac None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:57.185658 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf2227d-edf5-410b-a35b-d07eae77a1ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:57.185770 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c3a1b0-73f5-4d4d-9927-4b77ad4f896c None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:57.186076 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c3a1b0-73f5-4d4d-9927-4b77ad4f896c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:57.187542 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c3a1b0-73f5-4d4d-9927-4b77ad4f896c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:57.196297 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf2227d-edf5-410b-a35b-d07eae77a1ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:57.197751 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf2227d-edf5-410b-a35b-d07eae77a1ac None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:57.214399 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-3cf2227d-edf5-410b-a35b-d07eae77a1ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:44:57.214399 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:44:57.214399 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:44:57.214399 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:44:57.214399 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:44:57.214399 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:44:57.214399 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:44:57.214399 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:44:57.214399 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:44:57.214399 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:44:57.214399 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:44:57.214399 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:44:57.214399 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 10 12:44:57.214399 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 10 12:44:57.214399 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 10 12:44:57.215282 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 10 12:44:57.215282 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:44:57.215282 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:44:57.215282 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:44:57.215282 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:44:57.215282 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:44:57.215282 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:44:57.215282 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:44:57.215282 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:44:57.215282 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:44:57.215282 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:44:57.215282 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:44:57.215282 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:44:57.215282 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:44:57.215282 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:44:57.215282 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:44:57.215282 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:44:57.216174 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 10 12:44:57.216174 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 10 12:44:57.216174 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:44:57.216174 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:44:57.216174 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 257/497] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:44:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:44:57.216762 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677849b3-928f-4bd1-a88d-1cb6567cfc53 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:57.217028 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677849b3-928f-4bd1-a88d-1cb6567cfc53 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:57.217302 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677849b3-928f-4bd1-a88d-1cb6567cfc53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:57.224177 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 241/498] 10.222.0.27 () {56 vars in 1137 bytes} [Fri Jun 10 12:44:57 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:57.225009 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c3a1b0-73f5-4d4d-9927-4b77ad4f896c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:57.234528 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-677849b3-928f-4bd1-a88d-1cb6567cfc53 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:44:57.235735 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-677849b3-928f-4bd1-a88d-1cb6567cfc53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:44:57.245818 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:57.304198 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 258/499] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:44:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 10 12:44:57.305983 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35070909-fe93-4458-9232-1037b51cf4b8 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:57.306287 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35070909-fe93-4458-9232-1037b51cf4b8 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:57.306558 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35070909-fe93-4458-9232-1037b51cf4b8 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:57.308009 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 259/500] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:44:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:44:57.308529 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35070909-fe93-4458-9232-1037b51cf4b8 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:57.329145 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c3a1b0-73f5-4d4d-9927-4b77ad4f896c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:57.365254 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf883f9-a05e-42dd-bdde-c4088f93650a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:57.366043 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf883f9-a05e-42dd-bdde-c4088f93650a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:57.366330 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf883f9-a05e-42dd-bdde-c4088f93650a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:57.366618 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf883f9-a05e-42dd-bdde-c4088f93650a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:57.376457 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf883f9-a05e-42dd-bdde-c4088f93650a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:57.377914 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf883f9-a05e-42dd-bdde-c4088f93650a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:57.379607 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:57.395771 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-cdf883f9-a05e-42dd-bdde-c4088f93650a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:44:57.395771 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:44:57.395771 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:44:57.395771 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:44:57.395771 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:44:57.395771 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:44:57.395771 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:44:57.395771 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:44:57.395771 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:44:57.395771 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:44:57.395771 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:44:57.395771 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:44:57.395771 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 10 12:44:57.395771 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 10 12:44:57.395771 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 10 12:44:57.396757 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 10 12:44:57.396757 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:44:57.396757 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:44:57.396757 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:44:57.396757 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:44:57.396757 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:44:57.396757 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:44:57.396757 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:44:57.396757 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:44:57.396757 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:44:57.396757 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:44:57.396757 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:44:57.396757 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:44:57.396757 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:44:57.396757 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:44:57.396757 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:44:57.396757 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:44:57.397636 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 10 12:44:57.397636 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 10 12:44:57.397636 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:44:57.397636 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:44:57.397636 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 242/501] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:44:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:44:57.398056 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf883f9-a05e-42dd-bdde-c4088f93650a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:57.411714 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35070909-fe93-4458-9232-1037b51cf4b8 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:57.447379 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b75b905-c2fd-484a-80d2-e65f6f4c537f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:57.448065 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b75b905-c2fd-484a-80d2-e65f6f4c537f None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:57.448354 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b75b905-c2fd-484a-80d2-e65f6f4c537f None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:57.448634 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b75b905-c2fd-484a-80d2-e65f6f4c537f None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:57.449357 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b75b905-c2fd-484a-80d2-e65f6f4c537f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:57.449667 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b75b905-c2fd-484a-80d2-e65f6f4c537f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:57.451042 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b75b905-c2fd-484a-80d2-e65f6f4c537f None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:57.462497 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 260/502] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Jun 10 12:44:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:57.463657 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a0a784-b6ec-43b5-9585-60a07c84c665 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:57.463980 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a0a784-b6ec-43b5-9585-60a07c84c665 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:57.464268 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a0a784-b6ec-43b5-9585-60a07c84c665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:57.469929 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:57.480647 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-58a0a784-b6ec-43b5-9585-60a07c84c665 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:44:57.481829 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-58a0a784-b6ec-43b5-9585-60a07c84c665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:44:57.497661 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf883f9-a05e-42dd-bdde-c4088f93650a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:57.531501 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d916be5-4ce8-41b8-b37a-39b313eae4d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:57.532169 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d916be5-4ce8-41b8-b37a-39b313eae4d0 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:57.532451 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d916be5-4ce8-41b8-b37a-39b313eae4d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:57.532722 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d916be5-4ce8-41b8-b37a-39b313eae4d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:57.547698 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 261/503] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:44:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 10 12:44:57.548518 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a0a784-b6ec-43b5-9585-60a07c84c665 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:57.593617 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:57.600717 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d916be5-4ce8-41b8-b37a-39b313eae4d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:57.602857 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d916be5-4ce8-41b8-b37a-39b313eae4d0 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:57.655039 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a0a784-b6ec-43b5-9585-60a07c84c665 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:57.689650 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b65da28-8952-4c39-9afc-e43221acbeb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:57.690443 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b65da28-8952-4c39-9afc-e43221acbeb0 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:57.690730 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b65da28-8952-4c39-9afc-e43221acbeb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:57.691009 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b65da28-8952-4c39-9afc-e43221acbeb0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:57.692864 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:57.698935 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 243/504] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Jun 10 12:44:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 10 12:44:57.701740 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d916be5-4ce8-41b8-b37a-39b313eae4d0 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:57.706662 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b65da28-8952-4c39-9afc-e43221acbeb0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:44:57.714853 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-5b65da28-8952-4c39-9afc-e43221acbeb0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 10 12:44:57.714853 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:44:57.714853 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:44:57.714853 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:44:57.714853 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:44:57.714853 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:44:57.714853 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:44:57.714853 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:44:57.714853 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:44:57.714853 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:44:57.714853 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:44:57.714853 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:44:57.714853 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:44:57.714853 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:44:57.714853 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:44:57.714853 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:44:57.715746 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:44:57.715746 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:44:57.715746 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:44:57.715746 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:44:57.715746 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:44:57.715746 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:44:57.715746 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:44:57.715746 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:44:57.715746 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:44:57.715746 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:44:57.715746 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:44:57.715746 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:44:57.715746 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:44:57.715746 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:44:57.715746 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:44:57.715746 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:44:57.716601 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:44:57.716601 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:44:57.716601 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:44:57.716601 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:44:57.716601 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:44:57.716601 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:44:57.716601 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:44:57.716601 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:44:57.716601 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:44:57.716601 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:44:57.716601 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 10 12:44:57.716601 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:44:57.716601 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 262/505] 10.222.0.27 () {56 vars in 1102 bytes} [Fri Jun 10 12:44:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:44:57.717268 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b65da28-8952-4c39-9afc-e43221acbeb0 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:57.799066 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:57.805774 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d916be5-4ce8-41b8-b37a-39b313eae4d0 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:57.815433 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b65da28-8952-4c39-9afc-e43221acbeb0 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:57.841344 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45943855-dd90-432f-a6c7-a95200842df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:57.842010 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45943855-dd90-432f-a6c7-a95200842df9 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:57.842296 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45943855-dd90-432f-a6c7-a95200842df9 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:57.842575 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45943855-dd90-432f-a6c7-a95200842df9 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:57.843304 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45943855-dd90-432f-a6c7-a95200842df9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:57.843611 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45943855-dd90-432f-a6c7-a95200842df9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:57.845068 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45943855-dd90-432f-a6c7-a95200842df9 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:57.850462 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc74649e-2a73-4e5c-8f66-16adf522bdc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:57.851241 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc74649e-2a73-4e5c-8f66-16adf522bdc7 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:57.851528 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc74649e-2a73-4e5c-8f66-16adf522bdc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:57.851805 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc74649e-2a73-4e5c-8f66-16adf522bdc7 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:57.856834 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 244/506] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Jun 10 12:44:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:57.857654 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45943855-dd90-432f-a6c7-a95200842df9 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:57.861969 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc74649e-2a73-4e5c-8f66-16adf522bdc7 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:57.863415 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc74649e-2a73-4e5c-8f66-16adf522bdc7 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:57.879405 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-cc74649e-2a73-4e5c-8f66-16adf522bdc7 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 10 12:44:57.879405 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:44:57.879405 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:44:57.879405 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:44:57.879405 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:44:57.879405 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:44:57.879405 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:44:57.879405 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:44:57.879405 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:44:57.879405 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:44:57.879405 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:44:57.879405 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:44:57.879405 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 10 12:44:57.879405 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 10 12:44:57.879405 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 10 12:44:57.879405 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 10 12:44:57.880568 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:44:57.880568 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:44:57.880568 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 10 12:44:57.880568 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 10 12:44:57.880568 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 10 12:44:57.880568 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 10 12:44:57.880568 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:44:57.880568 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:44:57.880568 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:44:57.880568 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:44:57.880568 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:44:57.880568 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:44:57.880568 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:44:57.880568 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:44:57.880568 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:44:57.880568 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:44:57.880568 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:44:57.880568 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:44:57.881483 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:44:57.881483 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:44:57.881483 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 10 12:44:57.881483 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 10 12:44:57.881483 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 10 12:44:57.881483 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 10 12:44:57.881483 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 10 12:44:57.881483 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 10 12:44:57.881483 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 10 12:44:57.881483 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:44:57.881483 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 263/507] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Jun 10 12:44:57 2022] GET /identity/v3/users/cinder => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:44:57.882048 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc74649e-2a73-4e5c-8f66-16adf522bdc7 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:57.936987 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:57.959982 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45943855-dd90-432f-a6c7-a95200842df9 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:57.980523 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc74649e-2a73-4e5c-8f66-16adf522bdc7 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:57.992847 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6230c4-dfc0-4370-ad23-e2ad0e445dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:57.993631 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6230c4-dfc0-4370-ad23-e2ad0e445dc1 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:57.993914 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6230c4-dfc0-4370-ad23-e2ad0e445dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:57.994189 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6230c4-dfc0-4370-ad23-e2ad0e445dc1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:58.004538 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6230c4-dfc0-4370-ad23-e2ad0e445dc1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:58.005993 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6230c4-dfc0-4370-ad23-e2ad0e445dc1 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:58.012563 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e88cac-1bf5-4d87-8651-9e909294ab80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:58.013340 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e88cac-1bf5-4d87-8651-9e909294ab80 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:58.013620 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e88cac-1bf5-4d87-8651-9e909294ab80 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:58.013892 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e88cac-1bf5-4d87-8651-9e909294ab80 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:58.023176 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-fe6230c4-dfc0-4370-ad23-e2ad0e445dc1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:44:58.023176 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:44:58.023176 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:44:58.023176 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:44:58.023176 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:44:58.023176 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:44:58.023176 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:44:58.023176 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:44:58.023176 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:44:58.023176 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:44:58.023176 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:44:58.023176 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:44:58.023176 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 10 12:44:58.023176 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 10 12:44:58.023176 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 10 12:44:58.024660 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 10 12:44:58.024660 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:44:58.024660 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:44:58.024660 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:44:58.024660 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:44:58.024660 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:44:58.024660 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:44:58.024660 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:44:58.024660 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:44:58.024660 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:44:58.024660 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:44:58.024660 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:44:58.024660 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:44:58.024660 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:44:58.024660 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:44:58.024660 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:44:58.024660 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:44:58.025537 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 10 12:44:58.025537 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 10 12:44:58.025537 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:44:58.025537 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:44:58.025759 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e88cac-1bf5-4d87-8651-9e909294ab80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:58.025827 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 245/508] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:44:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:44:58.025827 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6230c4-dfc0-4370-ad23-e2ad0e445dc1 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:58.026063 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e88cac-1bf5-4d87-8651-9e909294ab80 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:58.036416 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:58.045148 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-c7e88cac-1bf5-4d87-8651-9e909294ab80 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:44:58.045148 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:44:58.045148 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:44:58.045148 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:44:58.045148 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:44:58.045148 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:44:58.045148 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:44:58.045148 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:44:58.045148 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:44:58.045148 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:44:58.045148 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:44:58.045148 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:44:58.045148 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 10 12:44:58.045148 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 10 12:44:58.045148 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 10 12:44:58.046085 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 10 12:44:58.046085 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:44:58.046085 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:44:58.046085 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:44:58.046085 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:44:58.046085 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:44:58.046085 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:44:58.046085 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:44:58.046085 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:44:58.046085 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:44:58.046085 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:44:58.046085 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:44:58.046085 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:44:58.046085 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:44:58.046085 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:44:58.046085 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:44:58.046085 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:44:58.051509 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 10 12:44:58.051509 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 10 12:44:58.051509 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:44:58.051509 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:44:58.051509 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 264/509] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:44:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:44:58.051509 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e88cac-1bf5-4d87-8651-9e909294ab80 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:58.127926 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6230c4-dfc0-4370-ad23-e2ad0e445dc1 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:58.143989 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e88cac-1bf5-4d87-8651-9e909294ab80 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:58.150745 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:58.161137 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70c821c-5706-4089-9e01-dbacd6a0e81d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:58.162060 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70c821c-5706-4089-9e01-dbacd6a0e81d None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:58.162354 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70c821c-5706-4089-9e01-dbacd6a0e81d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:58.162631 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70c821c-5706-4089-9e01-dbacd6a0e81d None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:58.163382 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70c821c-5706-4089-9e01-dbacd6a0e81d None admin] RBAC: Adding query filter params (name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:58.163693 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70c821c-5706-4089-9e01-dbacd6a0e81d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:58.165135 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70c821c-5706-4089-9e01-dbacd6a0e81d None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:58.173251 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 246/510] 10.222.0.27 () {56 vars in 1103 bytes} [Fri Jun 10 12:44:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:58.182676 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70c821c-5706-4089-9e01-dbacd6a0e81d None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:58.182788 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4e0989-9a20-45b4-94e2-69f7264f4549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:58.182788 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4e0989-9a20-45b4-94e2-69f7264f4549 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:58.182788 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4e0989-9a20-45b4-94e2-69f7264f4549 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:58.183088 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4e0989-9a20-45b4-94e2-69f7264f4549 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:58.190567 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4e0989-9a20-45b4-94e2-69f7264f4549 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:58.192034 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4e0989-9a20-45b4-94e2-69f7264f4549 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:58.206084 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-3a4e0989-9a20-45b4-94e2-69f7264f4549 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 10 12:44:58.206084 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:44:58.206084 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:44:58.206084 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:44:58.206084 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:44:58.206084 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:44:58.206084 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:44:58.206084 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:44:58.206084 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:44:58.206084 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:44:58.206084 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:44:58.206084 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:44:58.206084 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 10 12:44:58.206084 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 10 12:44:58.206084 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 10 12:44:58.206084 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 10 12:44:58.206984 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:44:58.206984 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:44:58.206984 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 10 12:44:58.206984 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 10 12:44:58.206984 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 10 12:44:58.206984 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 10 12:44:58.206984 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:44:58.206984 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:44:58.206984 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:44:58.206984 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:44:58.206984 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:44:58.206984 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:44:58.206984 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:44:58.206984 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:44:58.206984 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:44:58.206984 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:44:58.206984 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:44:58.206984 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:44:58.207919 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:44:58.207919 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:44:58.207919 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 10 12:44:58.207919 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 10 12:44:58.207919 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 10 12:44:58.207919 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 10 12:44:58.207919 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 10 12:44:58.207919 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 10 12:44:58.207919 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 10 12:44:58.207919 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:44:58.207919 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 265/511] 10.222.0.27 () {56 vars in 1093 bytes} [Fri Jun 10 12:44:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:44:58.208514 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4e0989-9a20-45b4-94e2-69f7264f4549 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:58.238607 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:58.271919 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70c821c-5706-4089-9e01-dbacd6a0e81d None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:58.304749 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9522f6-8f18-4056-9bf1-a805e2650394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:58.305943 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9522f6-8f18-4056-9bf1-a805e2650394 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:58.306223 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9522f6-8f18-4056-9bf1-a805e2650394 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:58.306494 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9522f6-8f18-4056-9bf1-a805e2650394 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:58.307321 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9522f6-8f18-4056-9bf1-a805e2650394 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:58.307624 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9522f6-8f18-4056-9bf1-a805e2650394 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:58.308670 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4e0989-9a20-45b4-94e2-69f7264f4549 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:58.309083 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9522f6-8f18-4056-9bf1-a805e2650394 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:58.326089 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 247/512] 10.222.0.27 () {56 vars in 1137 bytes} [Fri Jun 10 12:44:58 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:58.326869 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9522f6-8f18-4056-9bf1-a805e2650394 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:58.347772 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fd0139-d94f-4bcf-9fa2-d8beff8156fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:58.348476 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fd0139-d94f-4bcf-9fa2-d8beff8156fe None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:58.348764 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fd0139-d94f-4bcf-9fa2-d8beff8156fe None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:58.349045 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fd0139-d94f-4bcf-9fa2-d8beff8156fe None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:58.349769 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fd0139-d94f-4bcf-9fa2-d8beff8156fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:58.350080 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fd0139-d94f-4bcf-9fa2-d8beff8156fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:58.351449 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fd0139-d94f-4bcf-9fa2-d8beff8156fe None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:58.363300 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 266/513] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Jun 10 12:44:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:58.364103 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fd0139-d94f-4bcf-9fa2-d8beff8156fe None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:58.390919 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:58.425196 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9522f6-8f18-4056-9bf1-a805e2650394 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:58.450173 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:58.458772 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358cb806-34a4-486d-ad89-dc8e5c331a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:58.459620 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358cb806-34a4-486d-ad89-dc8e5c331a19 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:58.459936 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358cb806-34a4-486d-ad89-dc8e5c331a19 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:58.460221 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358cb806-34a4-486d-ad89-dc8e5c331a19 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:58.460944 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358cb806-34a4-486d-ad89-dc8e5c331a19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:58.461254 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358cb806-34a4-486d-ad89-dc8e5c331a19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:58.461543 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fd0139-d94f-4bcf-9fa2-d8beff8156fe None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:58.462659 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358cb806-34a4-486d-ad89-dc8e5c331a19 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:58.473429 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 248/514] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Jun 10 12:44:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:58.474205 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358cb806-34a4-486d-ad89-dc8e5c331a19 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:58.499117 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14325b68-c952-43a7-8d67-f0c362d530f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:58.499933 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14325b68-c952-43a7-8d67-f0c362d530f9 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:58.500221 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14325b68-c952-43a7-8d67-f0c362d530f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:58.500501 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14325b68-c952-43a7-8d67-f0c362d530f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:58.510343 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14325b68-c952-43a7-8d67-f0c362d530f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:58.511776 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14325b68-c952-43a7-8d67-f0c362d530f9 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:58.528998 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-14325b68-c952-43a7-8d67-f0c362d530f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:44:58.528998 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:44:58.528998 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:44:58.528998 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:44:58.528998 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:44:58.528998 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:44:58.528998 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:44:58.528998 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:44:58.528998 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:44:58.528998 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:44:58.528998 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:44:58.528998 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:44:58.528998 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 10 12:44:58.528998 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 10 12:44:58.528998 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 10 12:44:58.529844 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 10 12:44:58.529844 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:44:58.529844 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:44:58.529844 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:44:58.529844 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:44:58.529844 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:44:58.529844 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:44:58.529844 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:44:58.529844 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:44:58.529844 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:44:58.529844 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:44:58.529844 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:44:58.529844 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:44:58.529844 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:44:58.529844 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:44:58.529844 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:44:58.529844 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:44:58.530700 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 10 12:44:58.530700 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 10 12:44:58.530700 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:44:58.530700 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:44:58.530700 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 267/515] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:44:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:44:58.530961 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14325b68-c952-43a7-8d67-f0c362d530f9 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:58.575420 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358cb806-34a4-486d-ad89-dc8e5c331a19 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:58.609290 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f0aaed-6868-4e98-aeb8-167fa47003f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:58.610350 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f0aaed-6868-4e98-aeb8-167fa47003f2 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:58.610634 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f0aaed-6868-4e98-aeb8-167fa47003f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:58.610910 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f0aaed-6868-4e98-aeb8-167fa47003f2 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:58.611739 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f0aaed-6868-4e98-aeb8-167fa47003f2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:58.612075 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f0aaed-6868-4e98-aeb8-167fa47003f2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:58.613018 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:58.613503 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f0aaed-6868-4e98-aeb8-167fa47003f2 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:58.630745 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 249/516] 10.222.0.27 () {56 vars in 1133 bytes} [Fri Jun 10 12:44:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:58.631593 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f0aaed-6868-4e98-aeb8-167fa47003f2 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:58.638801 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14325b68-c952-43a7-8d67-f0c362d530f9 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:58.673551 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bb160f-ee67-4ca6-93fc-b19bdabfb2af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:58.674127 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bb160f-ee67-4ca6-93fc-b19bdabfb2af None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:58.674411 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bb160f-ee67-4ca6-93fc-b19bdabfb2af None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:58.674688 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bb160f-ee67-4ca6-93fc-b19bdabfb2af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:58.675628 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:58.728099 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f0aaed-6868-4e98-aeb8-167fa47003f2 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:58.741545 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bb160f-ee67-4ca6-93fc-b19bdabfb2af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:58.743614 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bb160f-ee67-4ca6-93fc-b19bdabfb2af None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:58.761157 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324fd5ff-add1-4ac7-b967-73aa5d0890b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:58.761821 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324fd5ff-add1-4ac7-b967-73aa5d0890b9 None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:58.762110 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324fd5ff-add1-4ac7-b967-73aa5d0890b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:58.762396 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324fd5ff-add1-4ac7-b967-73aa5d0890b9 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:58.763104 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324fd5ff-add1-4ac7-b967-73aa5d0890b9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:58.764562 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324fd5ff-add1-4ac7-b967-73aa5d0890b9 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:58.820584 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:58.841622 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 268/517] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Jun 10 12:44:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 10 12:44:58.842410 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bb160f-ee67-4ca6-93fc-b19bdabfb2af None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:58.851693 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 250/518] 10.222.0.27 () {60 vars in 1132 bytes} [Fri Jun 10 12:44:58 2022] POST /identity/v3/users => generated 247 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 12:44:58.853541 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324fd5ff-add1-4ac7-b967-73aa5d0890b9 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:58.958289 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bb160f-ee67-4ca6-93fc-b19bdabfb2af None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:58.962809 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:58.966402 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324fd5ff-add1-4ac7-b967-73aa5d0890b9 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:58.998967 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe06e3ef-771f-45fa-b7f6-e9758100167f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:58.999846 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe06e3ef-771f-45fa-b7f6-e9758100167f None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:59.000151 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe06e3ef-771f-45fa-b7f6-e9758100167f None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:59.000420 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe06e3ef-771f-45fa-b7f6-e9758100167f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:59.006235 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a289ee42-bc2a-4ad2-8fd1-2209a845fde4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:59.007068 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a289ee42-bc2a-4ad2-8fd1-2209a845fde4 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:59.007425 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a289ee42-bc2a-4ad2-8fd1-2209a845fde4 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:59.007641 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe06e3ef-771f-45fa-b7f6-e9758100167f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:59.007835 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a289ee42-bc2a-4ad2-8fd1-2209a845fde4 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:59.008704 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a289ee42-bc2a-4ad2-8fd1-2209a845fde4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:59.009089 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a289ee42-bc2a-4ad2-8fd1-2209a845fde4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:59.009235 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe06e3ef-771f-45fa-b7f6-e9758100167f None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:59.010575 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a289ee42-bc2a-4ad2-8fd1-2209a845fde4 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:59.024935 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-fe06e3ef-771f-45fa-b7f6-e9758100167f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 10 12:44:59.024935 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:44:59.024935 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:44:59.024935 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:44:59.024935 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:44:59.024935 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:44:59.024935 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:44:59.024935 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:44:59.024935 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:44:59.024935 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:44:59.024935 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:44:59.024935 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:44:59.024935 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 10 12:44:59.024935 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 10 12:44:59.024935 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 10 12:44:59.026004 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 10 12:44:59.026004 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:44:59.026004 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:44:59.026004 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:44:59.026004 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:44:59.026004 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:44:59.026004 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:44:59.026004 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:44:59.026004 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:44:59.026004 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:44:59.026004 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:44:59.026004 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:44:59.026004 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:44:59.026004 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:44:59.026004 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:44:59.026004 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:44:59.026004 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:44:59.026923 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 10 12:44:59.026923 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 10 12:44:59.026923 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 10 12:44:59.026923 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 10 12:44:59.026923 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 10 12:44:59.026923 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 10 12:44:59.026923 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 10 12:44:59.026923 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:44:59.026923 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 269/519] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Jun 10 12:44:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:44:59.028423 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe06e3ef-771f-45fa-b7f6-e9758100167f None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:59.028521 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 251/520] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Jun 10 12:44:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:59.029375 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a289ee42-bc2a-4ad2-8fd1-2209a845fde4 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:59.074847 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:59.137908 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a289ee42-bc2a-4ad2-8fd1-2209a845fde4 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:59.138105 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe06e3ef-771f-45fa-b7f6-e9758100167f None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:59.173539 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3789344-67a7-4ffb-821d-586c1a0ed5b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:59.174336 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3789344-67a7-4ffb-821d-586c1a0ed5b8 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:59.174633 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3789344-67a7-4ffb-821d-586c1a0ed5b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:59.174911 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3789344-67a7-4ffb-821d-586c1a0ed5b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:59.179510 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53b2d0c-f04c-417e-8975-299ee50b2db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:59.180481 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53b2d0c-f04c-417e-8975-299ee50b2db5 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:59.180967 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53b2d0c-f04c-417e-8975-299ee50b2db5 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:59.181820 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53b2d0c-f04c-417e-8975-299ee50b2db5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:59.185157 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3789344-67a7-4ffb-821d-586c1a0ed5b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:59.186595 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3789344-67a7-4ffb-821d-586c1a0ed5b8 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:59.204347 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-e3789344-67a7-4ffb-821d-586c1a0ed5b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:44:59.204347 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:44:59.204347 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:44:59.204347 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:44:59.204347 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:44:59.204347 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:44:59.204347 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:44:59.204347 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:44:59.204347 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:44:59.204347 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:44:59.204347 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:44:59.204347 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:44:59.204347 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 10 12:44:59.204347 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 10 12:44:59.204347 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 10 12:44:59.205280 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 10 12:44:59.205280 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:44:59.205280 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:44:59.205280 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:44:59.205280 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:44:59.205280 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:44:59.205280 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:44:59.205280 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:44:59.205280 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:44:59.205280 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:44:59.205280 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:44:59.205280 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:44:59.205280 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:44:59.205280 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:44:59.205280 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:44:59.205280 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:44:59.205280 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:44:59.206353 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 10 12:44:59.206353 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 10 12:44:59.206353 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:44:59.206353 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:44:59.206353 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 252/521] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:44:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:44:59.206353 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3789344-67a7-4ffb-821d-586c1a0ed5b8 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:59.224783 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:59.263251 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53b2d0c-f04c-417e-8975-299ee50b2db5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:59.264738 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53b2d0c-f04c-417e-8975-299ee50b2db5 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:59.304719 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:59.313280 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3789344-67a7-4ffb-821d-586c1a0ed5b8 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:59.351560 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaf9402-c2dc-4e7d-adeb-13718be5c5eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:59.352478 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaf9402-c2dc-4e7d-adeb-13718be5c5eb None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:59.352614 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaf9402-c2dc-4e7d-adeb-13718be5c5eb None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:59.352837 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaf9402-c2dc-4e7d-adeb-13718be5c5eb None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:59.353628 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaf9402-c2dc-4e7d-adeb-13718be5c5eb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:59.353934 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaf9402-c2dc-4e7d-adeb-13718be5c5eb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:59.355451 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaf9402-c2dc-4e7d-adeb-13718be5c5eb None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:59.368810 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 270/522] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Jun 10 12:44:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 10 12:44:59.369511 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53b2d0c-f04c-417e-8975-299ee50b2db5 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:59.383122 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 253/523] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Jun 10 12:44:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:59.384541 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaf9402-c2dc-4e7d-adeb-13718be5c5eb None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:59.455493 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:59.483085 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53b2d0c-f04c-417e-8975-299ee50b2db5 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:59.487128 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaf9402-c2dc-4e7d-adeb-13718be5c5eb None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:59.516367 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78f9bd2-94d0-4586-bf04-4dc278862626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:59.517170 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78f9bd2-94d0-4586-bf04-4dc278862626 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:59.517461 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78f9bd2-94d0-4586-bf04-4dc278862626 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:59.517732 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78f9bd2-94d0-4586-bf04-4dc278862626 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:59.522981 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399af68a-4560-42fd-8aa9-c3f8e881dc05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:59.523646 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399af68a-4560-42fd-8aa9-c3f8e881dc05 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:59.524566 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399af68a-4560-42fd-8aa9-c3f8e881dc05 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:59.524852 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399af68a-4560-42fd-8aa9-c3f8e881dc05 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:59.525581 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399af68a-4560-42fd-8aa9-c3f8e881dc05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:59.525892 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399af68a-4560-42fd-8aa9-c3f8e881dc05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:59.526213 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78f9bd2-94d0-4586-bf04-4dc278862626 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:59.527275 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399af68a-4560-42fd-8aa9-c3f8e881dc05 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:59.527638 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78f9bd2-94d0-4586-bf04-4dc278862626 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:59.529495 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:59.544181 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 254/524] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Jun 10 12:44:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:59.544181 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399af68a-4560-42fd-8aa9-c3f8e881dc05 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:59.544414 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-e78f9bd2-94d0-4586-bf04-4dc278862626 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 10 12:44:59.544414 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:44:59.544414 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:44:59.544414 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:44:59.544414 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:44:59.544414 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:44:59.544414 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:44:59.544414 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:44:59.544414 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:44:59.544414 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:44:59.544414 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:44:59.544414 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:44:59.544414 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 10 12:44:59.544414 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 10 12:44:59.544414 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 10 12:44:59.544414 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 10 12:44:59.545360 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:44:59.545360 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:44:59.545360 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 10 12:44:59.545360 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 10 12:44:59.545360 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 10 12:44:59.545360 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 10 12:44:59.545360 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:44:59.545360 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:44:59.545360 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:44:59.545360 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:44:59.545360 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:44:59.545360 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:44:59.545360 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:44:59.545360 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:44:59.545360 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:44:59.545360 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:44:59.545360 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:44:59.545360 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:44:59.546269 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:44:59.546269 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:44:59.546269 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 10 12:44:59.546269 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 10 12:44:59.546269 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 10 12:44:59.546269 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 10 12:44:59.546269 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 10 12:44:59.546269 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 10 12:44:59.546269 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 10 12:44:59.546269 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:44:59.546269 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 271/525] 10.222.0.27 () {56 vars in 1102 bytes} [Fri Jun 10 12:44:59 2022] GET /identity/v3/users/neutron => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:44:59.546269 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78f9bd2-94d0-4586-bf04-4dc278862626 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:59.652255 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399af68a-4560-42fd-8aa9-c3f8e881dc05 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:59.655636 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78f9bd2-94d0-4586-bf04-4dc278862626 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:59.684976 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:59.690402 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ae88fc-c3e4-494a-8b93-c92f9fb0fc15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:59.691387 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ae88fc-c3e4-494a-8b93-c92f9fb0fc15 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:59.691742 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ae88fc-c3e4-494a-8b93-c92f9fb0fc15 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:59.692045 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ae88fc-c3e4-494a-8b93-c92f9fb0fc15 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:59.694095 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d828f7-1eca-471d-abfd-f8f2b50fe4cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:59.694824 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d828f7-1eca-471d-abfd-f8f2b50fe4cf None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:59.695184 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d828f7-1eca-471d-abfd-f8f2b50fe4cf None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:59.695527 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d828f7-1eca-471d-abfd-f8f2b50fe4cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:59.697050 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d828f7-1eca-471d-abfd-f8f2b50fe4cf None admin] RBAC: Adding query filter params (role.id=ea7e8f74523b4322857fb3f4307d171e, scope.project.id=73ea5b23a2d34544ab493f4c5e4034b8, user.id=6d3eb01a98274050b930616b98998c06) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:59.697431 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d828f7-1eca-471d-abfd-f8f2b50fe4cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:59.699007 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d828f7-1eca-471d-abfd-f8f2b50fe4cf None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:59.703016 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ae88fc-c3e4-494a-8b93-c92f9fb0fc15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:59.704501 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ae88fc-c3e4-494a-8b93-c92f9fb0fc15 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:59.716254 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 272/526] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Jun 10 12:44:59 2022] GET /identity/v3/role_assignments?user.id=6d3eb01a98274050b930616b98998c06&scope.project.id=73ea5b23a2d34544ab493f4c5e4034b8&role.id=ea7e8f74523b4322857fb3f4307d171e => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:59.716254 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d828f7-1eca-471d-abfd-f8f2b50fe4cf None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:59.724093 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-b1ae88fc-c3e4-494a-8b93-c92f9fb0fc15 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:44:59.724093 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:44:59.724093 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:44:59.724093 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:44:59.724093 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:44:59.724093 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:44:59.724093 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:44:59.724093 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:44:59.724093 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:44:59.724093 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:44:59.724093 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:44:59.724093 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:44:59.724093 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 10 12:44:59.724093 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 10 12:44:59.724093 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 10 12:44:59.725218 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 10 12:44:59.725218 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:44:59.725218 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:44:59.725218 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:44:59.725218 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:44:59.725218 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:44:59.725218 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:44:59.725218 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:44:59.725218 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:44:59.725218 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:44:59.725218 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:44:59.725218 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:44:59.725218 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:44:59.725218 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:44:59.725218 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:44:59.725218 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:44:59.725218 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:44:59.726274 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 10 12:44:59.726274 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 10 12:44:59.726274 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:44:59.726274 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:44:59.731258 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 255/527] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:44:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:44:59.732256 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ae88fc-c3e4-494a-8b93-c92f9fb0fc15 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:59.807912 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:44:59.861673 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ae88fc-c3e4-494a-8b93-c92f9fb0fc15 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:59.880077 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d828f7-1eca-471d-abfd-f8f2b50fe4cf None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:44:59.912181 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fac6c2d-4841-4450-a1f3-9569dec96709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:59.912861 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fac6c2d-4841-4450-a1f3-9569dec96709 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:59.913147 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fac6c2d-4841-4450-a1f3-9569dec96709 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:59.913430 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fac6c2d-4841-4450-a1f3-9569dec96709 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:59.914592 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fac6c2d-4841-4450-a1f3-9569dec96709 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:44:59.915286 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fac6c2d-4841-4450-a1f3-9569dec96709 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:59.917067 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fac6c2d-4841-4450-a1f3-9569dec96709 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:59.924038 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3cf99e-9769-4156-a841-cfc3c791c4e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:44:59.925415 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3cf99e-9769-4156-a841-cfc3c791c4e6 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:44:59.925899 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3cf99e-9769-4156-a841-cfc3c791c4e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:44:59.926459 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3cf99e-9769-4156-a841-cfc3c791c4e6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:44:59.935661 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3cf99e-9769-4156-a841-cfc3c791c4e6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:44:59.941562 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3cf99e-9769-4156-a841-cfc3c791c4e6 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:44:59.941711 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 256/528] 10.222.0.27 () {56 vars in 1139 bytes} [Fri Jun 10 12:44:59 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:44:59.941711 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fac6c2d-4841-4450-a1f3-9569dec96709 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:59.960979 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-ac3cf99e-9769-4156-a841-cfc3c791c4e6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 10 12:44:59.960979 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:44:59.960979 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:44:59.960979 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:44:59.960979 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:44:59.960979 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:44:59.960979 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:44:59.960979 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:44:59.960979 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:44:59.960979 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:44:59.960979 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:44:59.960979 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:44:59.960979 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 10 12:44:59.960979 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 10 12:44:59.960979 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 10 12:44:59.961923 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 10 12:44:59.961923 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:44:59.961923 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:44:59.961923 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:44:59.961923 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:44:59.961923 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:44:59.961923 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:44:59.961923 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:44:59.961923 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:44:59.961923 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:44:59.961923 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:44:59.961923 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:44:59.961923 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:44:59.961923 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:44:59.961923 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:44:59.961923 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:44:59.961923 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:44:59.962790 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 10 12:44:59.962790 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 10 12:44:59.962790 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 10 12:44:59.962790 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 10 12:44:59.962790 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 10 12:44:59.962790 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 10 12:44:59.962790 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 10 12:44:59.962790 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:44:59.965414 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 273/529] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Jun 10 12:44:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:44:59.966510 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3cf99e-9769-4156-a841-cfc3c791c4e6 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:44:59.998652 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:00.076587 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fac6c2d-4841-4450-a1f3-9569dec96709 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:00.096772 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3cf99e-9769-4156-a841-cfc3c791c4e6 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:00.124618 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49716cbd-1373-40d0-a2db-0629038fb248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:00.125447 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49716cbd-1373-40d0-a2db-0629038fb248 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:00.125617 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:00.125921 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49716cbd-1373-40d0-a2db-0629038fb248 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:00.126813 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49716cbd-1373-40d0-a2db-0629038fb248 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:00.128036 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49716cbd-1373-40d0-a2db-0629038fb248 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:00.128464 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49716cbd-1373-40d0-a2db-0629038fb248 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:00.130307 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49716cbd-1373-40d0-a2db-0629038fb248 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:00.139595 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1654aa20-67b7-4a1d-ae31-46cfd504398c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:00.140535 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1654aa20-67b7-4a1d-ae31-46cfd504398c None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:00.141448 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1654aa20-67b7-4a1d-ae31-46cfd504398c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:00.142052 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1654aa20-67b7-4a1d-ae31-46cfd504398c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:00.148023 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 257/530] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Jun 10 12:44:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:00.149135 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49716cbd-1373-40d0-a2db-0629038fb248 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:00.230609 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1654aa20-67b7-4a1d-ae31-46cfd504398c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:00.232272 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1654aa20-67b7-4a1d-ae31-46cfd504398c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:00.282584 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49716cbd-1373-40d0-a2db-0629038fb248 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:00.285023 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:00.328207 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc84480b-e17a-46fb-bc05-b1ca253ea2fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:00.329212 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc84480b-e17a-46fb-bc05-b1ca253ea2fd None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:00.329831 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc84480b-e17a-46fb-bc05-b1ca253ea2fd None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:00.330666 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc84480b-e17a-46fb-bc05-b1ca253ea2fd None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:00.331963 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc84480b-e17a-46fb-bc05-b1ca253ea2fd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:00.332475 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc84480b-e17a-46fb-bc05-b1ca253ea2fd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:00.334592 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc84480b-e17a-46fb-bc05-b1ca253ea2fd None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:00.357117 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 258/531] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Jun 10 12:45:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:00.357117 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc84480b-e17a-46fb-bc05-b1ca253ea2fd None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:00.366376 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 274/532] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Jun 10 12:44:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 10 12:45:00.367688 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1654aa20-67b7-4a1d-ae31-46cfd504398c None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:00.392301 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:00.484015 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc84480b-e17a-46fb-bc05-b1ca253ea2fd None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:00.486039 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1654aa20-67b7-4a1d-ae31-46cfd504398c None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:00.521891 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4f03dd-eb00-414f-8f2b-0e0c635cb504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:00.522906 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4f03dd-eb00-414f-8f2b-0e0c635cb504 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:00.523307 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4f03dd-eb00-414f-8f2b-0e0c635cb504 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:00.523942 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4f03dd-eb00-414f-8f2b-0e0c635cb504 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:00.530871 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4f03dd-eb00-414f-8f2b-0e0c635cb504 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:00.532658 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ba4f9e-3390-44a5-901e-343cb14e135e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:00.532793 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4f03dd-eb00-414f-8f2b-0e0c635cb504 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:00.533595 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ba4f9e-3390-44a5-901e-343cb14e135e None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:00.534036 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ba4f9e-3390-44a5-901e-343cb14e135e None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:00.534457 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ba4f9e-3390-44a5-901e-343cb14e135e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:00.536081 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ba4f9e-3390-44a5-901e-343cb14e135e None admin] RBAC: Adding query filter params (role.id=ea7e8f74523b4322857fb3f4307d171e, scope.project.id=73ea5b23a2d34544ab493f4c5e4034b8, user.id=c6cc95a563b941ac92ab283aa0d5a5d5) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:00.536560 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ba4f9e-3390-44a5-901e-343cb14e135e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:00.538403 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ba4f9e-3390-44a5-901e-343cb14e135e None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:00.551918 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-ca4f03dd-eb00-414f-8f2b-0e0c635cb504 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 10 12:45:00.551918 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:00.551918 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:00.551918 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:00.551918 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:00.551918 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:00.551918 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:00.551918 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:00.551918 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:00.551918 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:00.551918 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:00.551918 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:00.551918 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 10 12:45:00.551918 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 10 12:45:00.551918 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 10 12:45:00.552834 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 10 12:45:00.552834 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:00.552834 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:00.552834 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:00.552834 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:00.552834 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:00.552834 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:00.552834 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:00.552834 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:00.552834 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:00.552834 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:00.552834 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:00.552834 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:00.552834 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:00.552834 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:00.552834 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:00.552834 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:00.553703 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 10 12:45:00.553703 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 10 12:45:00.553703 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 10 12:45:00.553703 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 10 12:45:00.553703 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 10 12:45:00.553703 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 10 12:45:00.553703 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 10 12:45:00.553703 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:00.554706 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 259/534] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Jun 10 12:45:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:00.554890 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 275/534] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Jun 10 12:45:00 2022] GET /identity/v3/role_assignments?user.id=c6cc95a563b941ac92ab283aa0d5a5d5&scope.project.id=73ea5b23a2d34544ab493f4c5e4034b8&role.id=ea7e8f74523b4322857fb3f4307d171e => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:00.556303 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4f03dd-eb00-414f-8f2b-0e0c635cb504 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:00.556811 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ba4f9e-3390-44a5-901e-343cb14e135e None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:00.601166 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:00.730015 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:00.748854 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4f03dd-eb00-414f-8f2b-0e0c635cb504 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:00.755889 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ba4f9e-3390-44a5-901e-343cb14e135e None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:00.815377 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec3ebae-0423-417a-9df6-f370c730da7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:00.816652 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec3ebae-0423-417a-9df6-f370c730da7d None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:00.817159 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec3ebae-0423-417a-9df6-f370c730da7d None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:00.817358 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8888d9b1-b1df-44fc-be0c-4708f8bca7fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:00.817841 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec3ebae-0423-417a-9df6-f370c730da7d None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:00.818160 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8888d9b1-b1df-44fc-be0c-4708f8bca7fc None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:00.818451 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8888d9b1-b1df-44fc-be0c-4708f8bca7fc None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:00.818728 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8888d9b1-b1df-44fc-be0c-4708f8bca7fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:00.819470 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec3ebae-0423-417a-9df6-f370c730da7d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:00.820020 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec3ebae-0423-417a-9df6-f370c730da7d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:00.823112 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec3ebae-0423-417a-9df6-f370c730da7d None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:00.831271 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8888d9b1-b1df-44fc-be0c-4708f8bca7fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:00.832762 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8888d9b1-b1df-44fc-be0c-4708f8bca7fc None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:00.845881 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 276/535] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Jun 10 12:45:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:00.852603 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec3ebae-0423-417a-9df6-f370c730da7d None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:00.861041 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-8888d9b1-b1df-44fc-be0c-4708f8bca7fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:00.861041 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:00.861041 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:00.861041 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:00.861041 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:00.861041 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:00.861041 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:00.861041 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:00.861041 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:00.861041 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:00.861041 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:00.861041 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:00.861041 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 10 12:45:00.861041 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 10 12:45:00.861041 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 10 12:45:00.862041 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 10 12:45:00.862041 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:00.862041 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:00.862041 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:00.862041 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:00.862041 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:00.862041 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:00.862041 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:00.862041 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:00.862041 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:00.862041 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:00.862041 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:00.862041 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:00.862041 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:00.862041 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:00.862041 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:00.862041 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:00.862899 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 10 12:45:00.862899 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 10 12:45:00.862899 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:00.862899 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:00.868517 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 260/536] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:45:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:00.869714 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8888d9b1-b1df-44fc-be0c-4708f8bca7fc None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:00.987073 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:01.021447 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8888d9b1-b1df-44fc-be0c-4708f8bca7fc None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:01.059121 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab34de6d-b519-4d0b-a80e-f46633e8ba79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:01.060192 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab34de6d-b519-4d0b-a80e-f46633e8ba79 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:01.060703 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab34de6d-b519-4d0b-a80e-f46633e8ba79 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:01.061179 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab34de6d-b519-4d0b-a80e-f46633e8ba79 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:01.062215 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab34de6d-b519-4d0b-a80e-f46633e8ba79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:01.062801 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab34de6d-b519-4d0b-a80e-f46633e8ba79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:01.064575 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab34de6d-b519-4d0b-a80e-f46633e8ba79 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:01.077527 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 261/537] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Jun 10 12:45:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:01.083931 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec3ebae-0423-417a-9df6-f370c730da7d None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:01.085436 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab34de6d-b519-4d0b-a80e-f46633e8ba79 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:01.118898 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61635171-f277-4a2a-9914-96aa399c01dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:01.119563 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61635171-f277-4a2a-9914-96aa399c01dd None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:01.119849 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61635171-f277-4a2a-9914-96aa399c01dd None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:01.120167 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61635171-f277-4a2a-9914-96aa399c01dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:01.121004 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61635171-f277-4a2a-9914-96aa399c01dd None admin] RBAC: Adding query filter params (role.id=ea7e8f74523b4322857fb3f4307d171e, scope.project.id=73ea5b23a2d34544ab493f4c5e4034b8, user.id=8bd6caa461e84100a3d5fe5af5d80f3e) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:01.121318 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61635171-f277-4a2a-9914-96aa399c01dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:01.123238 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61635171-f277-4a2a-9914-96aa399c01dd None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:01.135953 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 277/538] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Jun 10 12:45:00 2022] GET /identity/v3/role_assignments?user.id=8bd6caa461e84100a3d5fe5af5d80f3e&scope.project.id=73ea5b23a2d34544ab493f4c5e4034b8&role.id=ea7e8f74523b4322857fb3f4307d171e => generated 259 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 12:45:01.139562 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc754f86-2190-456d-b06e-6d8c466c1d83 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:01.139878 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc754f86-2190-456d-b06e-6d8c466c1d83 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:01.140188 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc754f86-2190-456d-b06e-6d8c466c1d83 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:01.145998 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 278/539] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:45:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 10 12:45:01.147564 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f39d637-a608-4b5c-9672-e595885a902e None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:01.147853 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f39d637-a608-4b5c-9672-e595885a902e None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:01.148167 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f39d637-a608-4b5c-9672-e595885a902e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:01.193538 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-5f39d637-a608-4b5c-9672-e595885a902e None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:45:01.197930 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5f39d637-a608-4b5c-9672-e595885a902e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:45:01.207452 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:01.262739 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab34de6d-b519-4d0b-a80e-f46633e8ba79 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:01.305519 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e072c729-268e-463e-86e7-d729812157fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:01.306834 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e072c729-268e-463e-86e7-d729812157fe None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:01.307662 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e072c729-268e-463e-86e7-d729812157fe None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:01.308173 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e072c729-268e-463e-86e7-d729812157fe None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:01.310987 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:01.317784 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e072c729-268e-463e-86e7-d729812157fe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:01.319578 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e072c729-268e-463e-86e7-d729812157fe None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:01.338582 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-e072c729-268e-463e-86e7-d729812157fe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 10 12:45:01.338582 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:01.338582 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:01.338582 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:01.338582 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:01.338582 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:01.338582 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:01.338582 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:01.338582 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:01.338582 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:01.338582 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:01.338582 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:01.338582 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 10 12:45:01.338582 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 10 12:45:01.338582 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 10 12:45:01.339507 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 10 12:45:01.339507 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:01.339507 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:01.339507 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:01.339507 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:01.339507 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:01.339507 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:01.339507 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:01.339507 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:01.339507 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:01.339507 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:01.339507 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:01.339507 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:01.339507 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:01.339507 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:01.339507 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:01.339507 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:01.340426 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 10 12:45:01.340426 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 10 12:45:01.340426 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 10 12:45:01.340426 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 10 12:45:01.340426 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 10 12:45:01.340426 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 10 12:45:01.340426 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 10 12:45:01.340426 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:01.341992 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 262/540] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Jun 10 12:45:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:01.343381 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 279/541] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:45:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 10 12:45:01.348829 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e072c729-268e-463e-86e7-d729812157fe None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:01.369578 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f39d637-a608-4b5c-9672-e595885a902e None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:01.479077 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f39d637-a608-4b5c-9672-e595885a902e None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:01.516067 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0c6011-8091-497a-8ee0-e1ea85de65c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:01.517238 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0c6011-8091-497a-8ee0-e1ea85de65c1 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:01.517722 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0c6011-8091-497a-8ee0-e1ea85de65c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:01.518195 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0c6011-8091-497a-8ee0-e1ea85de65c1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:01.522666 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e072c729-268e-463e-86e7-d729812157fe None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:01.527649 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:01.531087 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0c6011-8091-497a-8ee0-e1ea85de65c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:45:01.539455 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-9a0c6011-8091-497a-8ee0-e1ea85de65c1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 10 12:45:01.539455 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:01.539455 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:01.539455 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:01.539455 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:01.539455 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:01.539455 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:01.539455 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:01.539455 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:01.539455 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:01.539455 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:01.539455 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:01.539455 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:45:01.539455 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:45:01.539455 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:45:01.539455 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:45:01.546616 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:45:01.546616 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:45:01.546616 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:45:01.546616 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:45:01.546616 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:01.546616 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:01.546616 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:01.546616 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:01.546616 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:01.546616 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:01.546616 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:01.546616 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:01.546616 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:01.546616 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:01.546616 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:01.546616 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:01.547419 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:01.547419 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:01.547419 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:01.547419 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:01.547419 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:45:01.547419 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:45:01.547419 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:45:01.547419 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:45:01.547419 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:45:01.547419 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:45:01.547419 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 10 12:45:01.547419 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:01.547419 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 280/542] 10.222.0.27 () {56 vars in 1102 bytes} [Fri Jun 10 12:45:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:01.549534 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0c6011-8091-497a-8ee0-e1ea85de65c1 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:01.590256 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ac1228-5db5-4dc4-962a-872d9e867ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:01.590914 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ac1228-5db5-4dc4-962a-872d9e867ad1 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:01.591205 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ac1228-5db5-4dc4-962a-872d9e867ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:01.591487 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ac1228-5db5-4dc4-962a-872d9e867ad1 None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:01.592320 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ac1228-5db5-4dc4-962a-872d9e867ad1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:01.592635 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ac1228-5db5-4dc4-962a-872d9e867ad1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:01.594081 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ac1228-5db5-4dc4-962a-872d9e867ad1 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:01.622970 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 263/543] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Jun 10 12:45:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 12:45:01.628803 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ac1228-5db5-4dc4-962a-872d9e867ad1 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:01.683418 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0c6011-8091-497a-8ee0-e1ea85de65c1 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:01.726489 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:01.729490 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130373da-9d42-4ac9-a52c-663318ab2136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:01.730476 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130373da-9d42-4ac9-a52c-663318ab2136 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:01.731950 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130373da-9d42-4ac9-a52c-663318ab2136 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:01.733054 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130373da-9d42-4ac9-a52c-663318ab2136 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:01.734545 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130373da-9d42-4ac9-a52c-663318ab2136 None admin] RBAC: Adding query filter params (name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:01.735146 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130373da-9d42-4ac9-a52c-663318ab2136 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:01.737698 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130373da-9d42-4ac9-a52c-663318ab2136 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:01.757539 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 281/544] 10.222.0.27 () {56 vars in 1103 bytes} [Fri Jun 10 12:45:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:01.767174 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130373da-9d42-4ac9-a52c-663318ab2136 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:01.803792 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ac1228-5db5-4dc4-962a-872d9e867ad1 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:01.877648 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a85220d-b191-4168-9216-4a1b7b18ff98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:01.878252 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a85220d-b191-4168-9216-4a1b7b18ff98 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:01.878535 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a85220d-b191-4168-9216-4a1b7b18ff98 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:01.878809 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a85220d-b191-4168-9216-4a1b7b18ff98 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:01.879655 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a85220d-b191-4168-9216-4a1b7b18ff98 None admin] RBAC: Adding query filter params (role.id=ea7e8f74523b4322857fb3f4307d171e, scope.project.id=73ea5b23a2d34544ab493f4c5e4034b8, user.id=bc4415461a974c24ad7105a97c7d915b) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:01.879993 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a85220d-b191-4168-9216-4a1b7b18ff98 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:01.881422 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a85220d-b191-4168-9216-4a1b7b18ff98 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:01.887845 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:01.892033 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 264/545] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Jun 10 12:45:01 2022] GET /identity/v3/role_assignments?user.id=bc4415461a974c24ad7105a97c7d915b&scope.project.id=73ea5b23a2d34544ab493f4c5e4034b8&role.id=ea7e8f74523b4322857fb3f4307d171e => generated 259 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:01.972167 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130373da-9d42-4ac9-a52c-663318ab2136 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:02.027188 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8ae300-a0e9-4db8-acfd-c97476017018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:02.028049 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8ae300-a0e9-4db8-acfd-c97476017018 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:02.028338 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8ae300-a0e9-4db8-acfd-c97476017018 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:02.028615 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8ae300-a0e9-4db8-acfd-c97476017018 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:02.049127 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8ae300-a0e9-4db8-acfd-c97476017018 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:02.050674 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8ae300-a0e9-4db8-acfd-c97476017018 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:02.073371 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb59bc5-ac16-468b-9dc4-b9b0b800852b None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:02.073676 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb59bc5-ac16-468b-9dc4-b9b0b800852b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:02.073934 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb59bc5-ac16-468b-9dc4-b9b0b800852b None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:02.075530 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 265/546] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:45:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:45:02.076018 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-ef8ae300-a0e9-4db8-acfd-c97476017018 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:02.076018 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:02.076018 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:02.076018 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:02.076018 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:02.076018 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:02.076018 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:02.076018 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:02.076018 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:02.076018 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:02.076018 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:02.076018 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:02.076018 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 10 12:45:02.076018 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 10 12:45:02.076018 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 10 12:45:02.077080 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 10 12:45:02.077080 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:02.077080 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:02.077080 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:02.077080 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:02.077080 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:02.077080 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:02.077080 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:02.077080 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:02.077080 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:02.077080 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:02.077080 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:02.077080 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:02.077080 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:02.077080 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:02.077080 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:02.077080 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:02.078411 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 10 12:45:02.078411 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 10 12:45:02.078411 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:02.078411 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:02.078411 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 282/547] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:45:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:02.080917 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47845410-7ec4-464a-b0b2-d6cf1fa4b8e8 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:02.081212 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47845410-7ec4-464a-b0b2-d6cf1fa4b8e8 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:02.081495 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47845410-7ec4-464a-b0b2-d6cf1fa4b8e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:02.082684 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8ae300-a0e9-4db8-acfd-c97476017018 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:02.119399 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-47845410-7ec4-464a-b0b2-d6cf1fa4b8e8 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:45:02.120665 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-47845410-7ec4-464a-b0b2-d6cf1fa4b8e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:45:02.233774 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8ae300-a0e9-4db8-acfd-c97476017018 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:02.252225 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:02.274511 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaae1e8c-0059-4974-aae0-b81a8dee4bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:02.275286 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaae1e8c-0059-4974-aae0-b81a8dee4bda None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:02.275810 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaae1e8c-0059-4974-aae0-b81a8dee4bda None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:02.275992 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaae1e8c-0059-4974-aae0-b81a8dee4bda None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:02.276637 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaae1e8c-0059-4974-aae0-b81a8dee4bda None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:02.276947 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaae1e8c-0059-4974-aae0-b81a8dee4bda None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:02.278331 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaae1e8c-0059-4974-aae0-b81a8dee4bda None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:02.305714 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 283/548] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Jun 10 12:45:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 12:45:02.312896 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaae1e8c-0059-4974-aae0-b81a8dee4bda None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:02.320916 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 266/549] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:45:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 10 12:45:02.377803 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47845410-7ec4-464a-b0b2-d6cf1fa4b8e8 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:02.440006 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaae1e8c-0059-4974-aae0-b81a8dee4bda None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:02.483461 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d8eb16-21b3-430a-ad92-c2dc2b19ed8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:02.484285 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:02.484395 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d8eb16-21b3-430a-ad92-c2dc2b19ed8c None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:02.484587 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d8eb16-21b3-430a-ad92-c2dc2b19ed8c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:02.484844 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d8eb16-21b3-430a-ad92-c2dc2b19ed8c None admin] PATH_INFO: `/v3/users/swift` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:02.498004 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d8eb16-21b3-430a-ad92-c2dc2b19ed8c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:02.499473 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d8eb16-21b3-430a-ad92-c2dc2b19ed8c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:02.520018 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-81d8eb16-21b3-430a-ad92-c2dc2b19ed8c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 10 12:45:02.520018 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:02.520018 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:02.520018 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:02.520018 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:02.520018 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:02.520018 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:02.520018 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:02.520018 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:02.520018 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:02.520018 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:02.520018 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:02.520018 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 10 12:45:02.520018 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 10 12:45:02.520018 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 10 12:45:02.520018 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 10 12:45:02.521125 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:02.521125 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:02.521125 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 10 12:45:02.521125 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 10 12:45:02.521125 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 10 12:45:02.521125 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 10 12:45:02.521125 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:02.521125 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:02.521125 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:02.521125 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:02.521125 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:02.521125 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:02.521125 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:02.521125 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:02.521125 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:02.521125 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:02.521125 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:02.521125 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:02.522276 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:02.522276 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:02.522276 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 10 12:45:02.522276 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 10 12:45:02.522276 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 10 12:45:02.522276 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 10 12:45:02.522276 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 10 12:45:02.522276 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 10 12:45:02.522276 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 10 12:45:02.522276 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:02.524879 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 284/550] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Jun 10 12:45:02 2022] GET /identity/v3/users/swift => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:02.533913 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d8eb16-21b3-430a-ad92-c2dc2b19ed8c None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:02.596244 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:02.598635 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47845410-7ec4-464a-b0b2-d6cf1fa4b8e8 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:02.668977 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfd6daa-f330-4f53-99a3-6b87019a13d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:02.670023 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfd6daa-f330-4f53-99a3-6b87019a13d7 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:02.670313 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfd6daa-f330-4f53-99a3-6b87019a13d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:02.670590 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfd6daa-f330-4f53-99a3-6b87019a13d7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:02.688915 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfd6daa-f330-4f53-99a3-6b87019a13d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:45:02.700434 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-7bfd6daa-f330-4f53-99a3-6b87019a13d7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 10 12:45:02.700434 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:02.700434 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:02.700434 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:02.700434 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:02.700434 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:02.700434 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:02.700434 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:02.700434 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:02.700434 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:02.700434 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:02.700434 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:02.700434 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:45:02.700434 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:45:02.700434 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:45:02.700434 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:45:02.701572 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:45:02.701572 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:45:02.701572 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:45:02.701572 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:45:02.701572 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:02.701572 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:02.701572 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:02.701572 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:02.701572 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:02.701572 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:02.701572 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:02.701572 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:02.701572 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:02.701572 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:02.701572 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:02.701572 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:02.702420 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:02.702420 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:02.702420 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:02.702420 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:02.702420 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:45:02.702420 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:45:02.702420 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:45:02.702420 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:45:02.702420 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:45:02.702420 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:45:02.702420 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 10 12:45:02.702420 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:02.707175 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d8eb16-21b3-430a-ad92-c2dc2b19ed8c None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:02.708664 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 267/551] 10.222.0.27 () {56 vars in 1102 bytes} [Fri Jun 10 12:45:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:02.710472 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfd6daa-f330-4f53-99a3-6b87019a13d7 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:02.770683 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ee1f8b-6dba-4d16-8c8e-76ea3b9bca20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:02.771282 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ee1f8b-6dba-4d16-8c8e-76ea3b9bca20 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:02.771561 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ee1f8b-6dba-4d16-8c8e-76ea3b9bca20 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:02.771842 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ee1f8b-6dba-4d16-8c8e-76ea3b9bca20 None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:02.772703 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ee1f8b-6dba-4d16-8c8e-76ea3b9bca20 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:02.773010 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ee1f8b-6dba-4d16-8c8e-76ea3b9bca20 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:02.774989 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ee1f8b-6dba-4d16-8c8e-76ea3b9bca20 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:02.872203 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 285/552] 10.222.0.27 () {56 vars in 1135 bytes} [Fri Jun 10 12:45:02 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:02.872627 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ee1f8b-6dba-4d16-8c8e-76ea3b9bca20 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:02.904404 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfd6daa-f330-4f53-99a3-6b87019a13d7 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:02.925917 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:02.982121 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add1337f-f156-4f24-b63c-7ee5d6e6e9c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:02.983091 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add1337f-f156-4f24-b63c-7ee5d6e6e9c6 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:02.983386 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add1337f-f156-4f24-b63c-7ee5d6e6e9c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:02.983669 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add1337f-f156-4f24-b63c-7ee5d6e6e9c6 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:02.984480 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add1337f-f156-4f24-b63c-7ee5d6e6e9c6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:02.984945 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add1337f-f156-4f24-b63c-7ee5d6e6e9c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:02.987144 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add1337f-f156-4f24-b63c-7ee5d6e6e9c6 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:03.003814 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 268/553] 10.222.0.27 () {56 vars in 1103 bytes} [Fri Jun 10 12:45:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 12:45:03.005510 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fe779c-651f-48bf-925d-3531d2b7cd9d None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:03.006227 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fe779c-651f-48bf-925d-3531d2b7cd9d None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:03.006702 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fe779c-651f-48bf-925d-3531d2b7cd9d None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:03.012213 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 269/554] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:45:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:45:03.013264 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fe779c-651f-48bf-925d-3531d2b7cd9d None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:03.016946 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:03.029072 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ee1f8b-6dba-4d16-8c8e-76ea3b9bca20 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:03.066296 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e4bef0-2553-4145-ab9a-4006787ef342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:03.067263 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e4bef0-2553-4145-ab9a-4006787ef342 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:03.067689 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e4bef0-2553-4145-ab9a-4006787ef342 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:03.068175 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e4bef0-2553-4145-ab9a-4006787ef342 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:03.124013 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fe779c-651f-48bf-925d-3531d2b7cd9d None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:03.165174 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e4bef0-2553-4145-ab9a-4006787ef342 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:03.166669 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e4bef0-2553-4145-ab9a-4006787ef342 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:03.172124 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39a3916-bc1f-426d-adbd-b70d82870d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:03.173189 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39a3916-bc1f-426d-adbd-b70d82870d84 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:03.173672 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39a3916-bc1f-426d-adbd-b70d82870d84 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:03.174116 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39a3916-bc1f-426d-adbd-b70d82870d84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:03.190505 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39a3916-bc1f-426d-adbd-b70d82870d84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:03.192847 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39a3916-bc1f-426d-adbd-b70d82870d84 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:03.217250 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-b39a3916-bc1f-426d-adbd-b70d82870d84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:03.217250 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:03.217250 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:03.217250 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:03.217250 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:03.217250 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:03.217250 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:03.217250 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:03.217250 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:03.217250 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:03.217250 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:03.217250 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:03.217250 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 10 12:45:03.217250 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 10 12:45:03.217250 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 10 12:45:03.218182 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 10 12:45:03.218182 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:03.218182 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:03.218182 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:03.218182 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:03.218182 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:03.218182 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:03.218182 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:03.218182 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:03.218182 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:03.218182 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:03.218182 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:03.218182 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:03.218182 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:03.218182 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:03.218182 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:03.218182 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:03.219055 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 10 12:45:03.219055 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 10 12:45:03.219055 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:03.219055 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:03.221738 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 270/555] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:45:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:03.223179 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a45382-3e10-4034-843f-1cb4f6616599 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:03.223619 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a45382-3e10-4034-843f-1cb4f6616599 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:03.224614 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a45382-3e10-4034-843f-1cb4f6616599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:03.249820 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-c9a45382-3e10-4034-843f-1cb4f6616599 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:45:03.251154 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c9a45382-3e10-4034-843f-1cb4f6616599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:45:03.270266 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:03.286488 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:03.295038 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 286/556] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Jun 10 12:45:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 10 12:45:03.296185 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e4bef0-2553-4145-ab9a-4006787ef342 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:03.343898 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 271/557] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:45:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 10 12:45:03.343898 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a45382-3e10-4034-843f-1cb4f6616599 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:03.429278 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e4bef0-2553-4145-ab9a-4006787ef342 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:03.466428 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942901f3-bb55-4c4a-9606-05b941d2e0a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:03.467097 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942901f3-bb55-4c4a-9606-05b941d2e0a7 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:03.467383 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942901f3-bb55-4c4a-9606-05b941d2e0a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:03.467664 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942901f3-bb55-4c4a-9606-05b941d2e0a7 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:03.468416 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942901f3-bb55-4c4a-9606-05b941d2e0a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:03.468731 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942901f3-bb55-4c4a-9606-05b941d2e0a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:03.470093 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942901f3-bb55-4c4a-9606-05b941d2e0a7 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:03.482624 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 287/558] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Jun 10 12:45:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:03.483154 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a45382-3e10-4034-843f-1cb4f6616599 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:03.483518 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942901f3-bb55-4c4a-9606-05b941d2e0a7 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:03.520912 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912a0782-f460-47b3-b6c7-5b710c1f6d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:03.522065 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912a0782-f460-47b3-b6c7-5b710c1f6d94 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:03.522578 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912a0782-f460-47b3-b6c7-5b710c1f6d94 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:03.523054 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912a0782-f460-47b3-b6c7-5b710c1f6d94 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:03.535390 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912a0782-f460-47b3-b6c7-5b710c1f6d94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:03.537468 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912a0782-f460-47b3-b6c7-5b710c1f6d94 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:03.538091 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:03.557884 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-912a0782-f460-47b3-b6c7-5b710c1f6d94 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:03.557884 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:03.557884 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:03.557884 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:03.557884 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:03.557884 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:03.557884 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:03.557884 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:03.557884 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:03.557884 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:03.557884 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:03.557884 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:03.557884 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 10 12:45:03.557884 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 10 12:45:03.557884 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 10 12:45:03.559301 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 10 12:45:03.559301 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:03.559301 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:03.559301 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:03.559301 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:03.559301 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:03.559301 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:03.559301 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:03.559301 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:03.559301 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:03.559301 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:03.559301 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:03.559301 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:03.559301 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:03.559301 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:03.559301 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:03.559301 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:03.560255 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 10 12:45:03.560255 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 10 12:45:03.560255 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:03.560255 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:03.562816 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 272/559] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:45:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:03.564076 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912a0782-f460-47b3-b6c7-5b710c1f6d94 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:03.568562 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:03.598035 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942901f3-bb55-4c4a-9606-05b941d2e0a7 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:03.635013 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58bdb46-4cec-494a-ad56-b20d0e0fad00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:03.635832 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58bdb46-4cec-494a-ad56-b20d0e0fad00 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:03.636200 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58bdb46-4cec-494a-ad56-b20d0e0fad00 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:03.636484 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58bdb46-4cec-494a-ad56-b20d0e0fad00 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:03.646990 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58bdb46-4cec-494a-ad56-b20d0e0fad00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:45:03.656023 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-f58bdb46-4cec-494a-ad56-b20d0e0fad00 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 10 12:45:03.656023 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:03.656023 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:03.656023 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:03.656023 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:03.656023 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:03.656023 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:03.656023 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:03.656023 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:03.656023 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:03.656023 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:03.656023 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:03.656023 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:45:03.656023 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:45:03.656023 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:45:03.656023 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:45:03.657003 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:45:03.657003 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:45:03.657003 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:45:03.657003 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:45:03.657003 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:03.657003 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:03.657003 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:03.657003 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:03.657003 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:03.657003 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:03.657003 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:03.657003 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:03.657003 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:03.657003 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:03.657003 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:03.657003 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:03.658516 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:03.658516 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:03.658516 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:03.658516 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:03.658516 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:45:03.658516 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:45:03.658516 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:45:03.658516 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:45:03.658516 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:45:03.658516 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:45:03.658516 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 10 12:45:03.658516 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:03.658516 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 288/560] 10.222.0.27 () {56 vars in 1102 bytes} [Fri Jun 10 12:45:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:03.658516 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58bdb46-4cec-494a-ad56-b20d0e0fad00 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:03.693761 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912a0782-f460-47b3-b6c7-5b710c1f6d94 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:03.730677 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce4af97-a1f9-44a2-a96f-3cb4e24ee3a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:03.731478 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce4af97-a1f9-44a2-a96f-3cb4e24ee3a4 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:03.731762 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce4af97-a1f9-44a2-a96f-3cb4e24ee3a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:03.732073 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce4af97-a1f9-44a2-a96f-3cb4e24ee3a4 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:03.740311 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce4af97-a1f9-44a2-a96f-3cb4e24ee3a4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:03.741763 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce4af97-a1f9-44a2-a96f-3cb4e24ee3a4 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:03.757528 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-cce4af97-a1f9-44a2-a96f-3cb4e24ee3a4 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 10 12:45:03.757528 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:03.757528 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:03.757528 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:03.757528 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:03.757528 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:03.757528 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:03.757528 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:03.757528 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:03.757528 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:03.757528 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:03.757528 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:03.757528 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 10 12:45:03.757528 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 10 12:45:03.757528 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 10 12:45:03.757528 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 10 12:45:03.759281 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:03.759281 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:03.759281 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 10 12:45:03.759281 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 10 12:45:03.759281 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 10 12:45:03.759281 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 10 12:45:03.759281 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:03.759281 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:03.759281 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:03.759281 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:03.759281 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:03.759281 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:03.759281 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:03.759281 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:03.759281 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:03.759281 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:03.759281 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:03.759281 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:03.760612 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:03.760612 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:03.760612 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 10 12:45:03.760612 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 10 12:45:03.760612 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 10 12:45:03.760612 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 10 12:45:03.760612 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 10 12:45:03.760612 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 10 12:45:03.760612 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 10 12:45:03.760612 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:03.763972 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 273/561] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Jun 10 12:45:03 2022] GET /identity/v3/users/glance => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:03.765294 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9749cab9-4fff-458e-9e49-d5c706b5db32 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:03.765590 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9749cab9-4fff-458e-9e49-d5c706b5db32 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:03.765876 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9749cab9-4fff-458e-9e49-d5c706b5db32 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:03.767152 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 274/562] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:45:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:45:03.767837 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9749cab9-4fff-458e-9e49-d5c706b5db32 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:03.775717 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58bdb46-4cec-494a-ad56-b20d0e0fad00 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:03.791586 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:03.813571 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aab51c0-6f89-46ab-8508-b8175f122ef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:03.814663 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aab51c0-6f89-46ab-8508-b8175f122ef7 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:03.814956 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aab51c0-6f89-46ab-8508-b8175f122ef7 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:03.815243 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aab51c0-6f89-46ab-8508-b8175f122ef7 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:03.816360 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aab51c0-6f89-46ab-8508-b8175f122ef7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:03.816678 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aab51c0-6f89-46ab-8508-b8175f122ef7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:03.818415 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aab51c0-6f89-46ab-8508-b8175f122ef7 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:03.837347 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 289/563] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Jun 10 12:45:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:03.837347 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aab51c0-6f89-46ab-8508-b8175f122ef7 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:03.842897 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:03.887910 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9749cab9-4fff-458e-9e49-d5c706b5db32 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:03.922735 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784c9b0f-1250-42db-8cdf-a5809c45a328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:03.923406 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784c9b0f-1250-42db-8cdf-a5809c45a328 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:03.923694 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784c9b0f-1250-42db-8cdf-a5809c45a328 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:03.924004 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784c9b0f-1250-42db-8cdf-a5809c45a328 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:03.924751 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784c9b0f-1250-42db-8cdf-a5809c45a328 None admin] RBAC: Adding query filter params (name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:03.925060 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784c9b0f-1250-42db-8cdf-a5809c45a328 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:03.926439 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784c9b0f-1250-42db-8cdf-a5809c45a328 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:03.934845 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 275/564] 10.222.0.27 () {56 vars in 1103 bytes} [Fri Jun 10 12:45:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:03.936095 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238079ff-fd14-4caf-a982-5d0e982d7df8 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:03.936385 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238079ff-fd14-4caf-a982-5d0e982d7df8 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:03.936667 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238079ff-fd14-4caf-a982-5d0e982d7df8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:03.950240 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aab51c0-6f89-46ab-8508-b8175f122ef7 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:03.953833 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-238079ff-fd14-4caf-a982-5d0e982d7df8 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:45:03.955028 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-238079ff-fd14-4caf-a982-5d0e982d7df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:45:03.986563 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60514701-2b75-438e-8e16-760cbd4c4cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:03.987201 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60514701-2b75-438e-8e16-760cbd4c4cf9 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:03.987487 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60514701-2b75-438e-8e16-760cbd4c4cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:03.987764 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60514701-2b75-438e-8e16-760cbd4c4cf9 None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:03.988666 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60514701-2b75-438e-8e16-760cbd4c4cf9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:03.988935 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60514701-2b75-438e-8e16-760cbd4c4cf9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:03.990381 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60514701-2b75-438e-8e16-760cbd4c4cf9 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:04.007506 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 290/565] 10.222.0.27 () {56 vars in 1137 bytes} [Fri Jun 10 12:45:03 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:04.008702 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60514701-2b75-438e-8e16-760cbd4c4cf9 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:04.039385 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 276/566] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:45:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 10 12:45:04.039385 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238079ff-fd14-4caf-a982-5d0e982d7df8 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:04.050298 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:04.099324 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:04.130110 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60514701-2b75-438e-8e16-760cbd4c4cf9 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:04.145850 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238079ff-fd14-4caf-a982-5d0e982d7df8 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:04.171959 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22612c5c-cae5-41aa-9f64-5430e4197444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:04.172732 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22612c5c-cae5-41aa-9f64-5430e4197444 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:04.173020 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22612c5c-cae5-41aa-9f64-5430e4197444 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:04.173299 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22612c5c-cae5-41aa-9f64-5430e4197444 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:04.181987 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22612c5c-cae5-41aa-9f64-5430e4197444 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:04.184121 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22612c5c-cae5-41aa-9f64-5430e4197444 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:04.191272 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e742d82c-4cc4-44d7-9459-78327cff46f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:04.192103 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e742d82c-4cc4-44d7-9459-78327cff46f4 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:04.192390 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e742d82c-4cc4-44d7-9459-78327cff46f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:04.192668 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e742d82c-4cc4-44d7-9459-78327cff46f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:04.200202 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-22612c5c-cae5-41aa-9f64-5430e4197444 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 10 12:45:04.200202 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:04.200202 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:04.200202 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:04.200202 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:04.200202 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:04.200202 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:04.200202 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:04.200202 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:04.200202 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:04.200202 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:04.200202 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:04.200202 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 10 12:45:04.200202 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 10 12:45:04.200202 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 10 12:45:04.207364 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 10 12:45:04.207364 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:04.207364 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:04.207364 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:04.207364 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:04.207364 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:04.207364 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:04.207364 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:04.207364 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:04.207364 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:04.207364 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:04.207364 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:04.207364 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:04.207364 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:04.207364 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:04.207364 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:04.207364 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:04.208675 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 10 12:45:04.208675 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 10 12:45:04.208675 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 10 12:45:04.208675 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 10 12:45:04.208675 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 10 12:45:04.208675 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 10 12:45:04.208675 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 10 12:45:04.208675 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:04.208675 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 291/567] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Jun 10 12:45:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:04.208675 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22612c5c-cae5-41aa-9f64-5430e4197444 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:04.209227 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e742d82c-4cc4-44d7-9459-78327cff46f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:04.209227 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e742d82c-4cc4-44d7-9459-78327cff46f4 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:04.225858 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-e742d82c-4cc4-44d7-9459-78327cff46f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:04.225858 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:04.225858 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:04.225858 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:04.225858 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:04.225858 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:04.225858 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:04.225858 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:04.225858 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:04.225858 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:04.225858 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:04.225858 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:04.225858 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 10 12:45:04.225858 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 10 12:45:04.225858 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 10 12:45:04.227001 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 10 12:45:04.227001 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:04.227001 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:04.227001 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:04.227001 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:04.227001 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:04.227001 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:04.227001 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:04.227001 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:04.227001 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:04.227001 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:04.227001 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:04.227001 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:04.227001 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:04.227001 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:04.227001 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:04.227001 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:04.232701 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 10 12:45:04.232701 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 10 12:45:04.232701 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:04.232701 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:04.232701 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 277/568] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:45:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:04.232701 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e742d82c-4cc4-44d7-9459-78327cff46f4 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:04.293112 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:04.303120 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22612c5c-cae5-41aa-9f64-5430e4197444 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:04.328239 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e742d82c-4cc4-44d7-9459-78327cff46f4 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:04.337430 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:04.338868 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647596aa-ff58-475d-b839-1ba15ce47054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:04.339664 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647596aa-ff58-475d-b839-1ba15ce47054 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:04.339989 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647596aa-ff58-475d-b839-1ba15ce47054 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:04.340270 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647596aa-ff58-475d-b839-1ba15ce47054 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:04.350525 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647596aa-ff58-475d-b839-1ba15ce47054 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:04.352016 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647596aa-ff58-475d-b839-1ba15ce47054 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:04.362135 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799c4057-6003-4424-bd7a-debfdc3d9dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:04.362932 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799c4057-6003-4424-bd7a-debfdc3d9dfc None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:04.363220 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799c4057-6003-4424-bd7a-debfdc3d9dfc None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:04.363500 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799c4057-6003-4424-bd7a-debfdc3d9dfc None admin] PATH_INFO: `/v3/roles/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:04.369604 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-647596aa-ff58-475d-b839-1ba15ce47054 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:04.369604 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:04.369604 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:04.369604 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:04.369604 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:04.369604 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:04.369604 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:04.369604 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:04.369604 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:04.369604 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:04.369604 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:04.369604 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:04.369604 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 10 12:45:04.369604 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 10 12:45:04.369604 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 10 12:45:04.370475 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 10 12:45:04.370475 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:04.370475 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:04.370475 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:04.370475 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:04.370475 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:04.370475 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:04.370475 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:04.370475 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:04.370475 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:04.370475 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:04.370475 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:04.370475 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:04.370475 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:04.370475 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:04.370475 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:04.370475 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:04.371384 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 10 12:45:04.371384 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 10 12:45:04.371384 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:04.371384 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:04.371384 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 292/569] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:45:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:04.371665 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647596aa-ff58-475d-b839-1ba15ce47054 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:04.375021 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799c4057-6003-4424-bd7a-debfdc3d9dfc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:45:04.382790 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-799c4057-6003-4424-bd7a-debfdc3d9dfc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 10 12:45:04.382790 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:04.382790 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:04.382790 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:04.382790 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:04.382790 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:04.382790 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:04.382790 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:04.382790 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:04.382790 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:04.382790 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:04.382790 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:04.382790 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:45:04.382790 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:45:04.382790 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:45:04.382790 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:45:04.384367 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:45:04.384367 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:45:04.384367 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:45:04.384367 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:45:04.384367 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:04.384367 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:04.384367 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:04.384367 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:04.384367 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:04.384367 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:04.384367 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:04.384367 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:04.384367 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:04.384367 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:04.384367 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:04.384367 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:04.385447 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:04.385447 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:04.385447 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:04.385447 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:04.385447 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:45:04.385447 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:45:04.385447 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:45:04.385447 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:45:04.385447 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:45:04.385447 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:45:04.385447 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 10 12:45:04.385447 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:04.385447 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 278/570] 10.222.0.27 () {56 vars in 1102 bytes} [Fri Jun 10 12:45:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:04.386437 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29db2663-729e-4056-b5d6-c00ddba43da0 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:04.386437 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29db2663-729e-4056-b5d6-c00ddba43da0 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:04.386676 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29db2663-729e-4056-b5d6-c00ddba43da0 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:04.387805 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 279/571] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:45:04 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:45:04.388520 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29db2663-729e-4056-b5d6-c00ddba43da0 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:04.479410 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647596aa-ff58-475d-b839-1ba15ce47054 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:04.487920 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29db2663-729e-4056-b5d6-c00ddba43da0 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:04.513364 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e425f92-a262-4080-ab9c-0c9266b68ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:04.514019 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e425f92-a262-4080-ab9c-0c9266b68ec5 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:04.514301 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e425f92-a262-4080-ab9c-0c9266b68ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:04.514582 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e425f92-a262-4080-ab9c-0c9266b68ec5 None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:04.515365 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e425f92-a262-4080-ab9c-0c9266b68ec5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:04.515679 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e425f92-a262-4080-ab9c-0c9266b68ec5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:04.517170 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e425f92-a262-4080-ab9c-0c9266b68ec5 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:04.520788 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17fbebc-57fb-4579-a8df-1d022bae6636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:04.521433 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17fbebc-57fb-4579-a8df-1d022bae6636 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:04.521718 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17fbebc-57fb-4579-a8df-1d022bae6636 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:04.522001 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17fbebc-57fb-4579-a8df-1d022bae6636 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:04.522715 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17fbebc-57fb-4579-a8df-1d022bae6636 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:04.523024 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17fbebc-57fb-4579-a8df-1d022bae6636 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:04.524404 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17fbebc-57fb-4579-a8df-1d022bae6636 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:04.529729 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 293/572] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Jun 10 12:45:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:04.530296 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e425f92-a262-4080-ab9c-0c9266b68ec5 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:04.532780 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:04.540485 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 280/573] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Jun 10 12:45:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:04.541287 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17fbebc-57fb-4579-a8df-1d022bae6636 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:04.569130 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:04.631743 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e425f92-a262-4080-ab9c-0c9266b68ec5 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:04.643359 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17fbebc-57fb-4579-a8df-1d022bae6636 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:04.667106 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa0e1f0-0162-4ad7-9e07-51a755ebdc64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:04.667766 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa0e1f0-0162-4ad7-9e07-51a755ebdc64 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:04.668075 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa0e1f0-0162-4ad7-9e07-51a755ebdc64 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:04.668353 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa0e1f0-0162-4ad7-9e07-51a755ebdc64 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:04.669067 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa0e1f0-0162-4ad7-9e07-51a755ebdc64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:04.669368 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa0e1f0-0162-4ad7-9e07-51a755ebdc64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:04.670765 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa0e1f0-0162-4ad7-9e07-51a755ebdc64 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:04.678492 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f322804e-5b19-44c8-800a-1f7a87cac6df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:04.679159 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f322804e-5b19-44c8-800a-1f7a87cac6df None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:04.679441 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f322804e-5b19-44c8-800a-1f7a87cac6df None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:04.679716 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f322804e-5b19-44c8-800a-1f7a87cac6df None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:04.681073 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f322804e-5b19-44c8-800a-1f7a87cac6df None admin] RBAC: Adding query filter params (name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:04.681371 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f322804e-5b19-44c8-800a-1f7a87cac6df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:04.682684 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 294/574] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Jun 10 12:45:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:04.682794 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f322804e-5b19-44c8-800a-1f7a87cac6df None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:04.683974 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42483f84-c390-42f2-88a5-ca64aca30793 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:04.684235 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42483f84-c390-42f2-88a5-ca64aca30793 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:04.684511 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42483f84-c390-42f2-88a5-ca64aca30793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:04.694925 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 281/575] 10.222.0.27 () {56 vars in 1103 bytes} [Fri Jun 10 12:45:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:04.694925 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f322804e-5b19-44c8-800a-1f7a87cac6df None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:04.700445 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-42483f84-c390-42f2-88a5-ca64aca30793 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:45:04.701621 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-42483f84-c390-42f2-88a5-ca64aca30793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:45:04.756334 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:04.766546 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 295/576] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:45:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 10 12:45:04.767345 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42483f84-c390-42f2-88a5-ca64aca30793 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:04.785171 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:04.795291 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f322804e-5b19-44c8-800a-1f7a87cac6df None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:04.829149 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0245fb2-0955-4c45-a9e0-01c68c2f54f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:04.829732 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0245fb2-0955-4c45-a9e0-01c68c2f54f6 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:04.830016 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0245fb2-0955-4c45-a9e0-01c68c2f54f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:04.830292 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0245fb2-0955-4c45-a9e0-01c68c2f54f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:04.831116 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0245fb2-0955-4c45-a9e0-01c68c2f54f6 None admin] RBAC: Adding query filter params (role.id=ea7e8f74523b4322857fb3f4307d171e, scope.project.id=73ea5b23a2d34544ab493f4c5e4034b8, user.id=fb63b9cd74bf438aa25b72cef8337fec) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:04.831426 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0245fb2-0955-4c45-a9e0-01c68c2f54f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:04.832939 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0245fb2-0955-4c45-a9e0-01c68c2f54f6 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:04.838886 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 282/577] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Jun 10 12:45:04 2022] GET /identity/v3/role_assignments?user.id=fb63b9cd74bf438aa25b72cef8337fec&scope.project.id=73ea5b23a2d34544ab493f4c5e4034b8&role.id=ea7e8f74523b4322857fb3f4307d171e => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:04.839652 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0245fb2-0955-4c45-a9e0-01c68c2f54f6 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:04.879360 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42483f84-c390-42f2-88a5-ca64aca30793 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:04.916642 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9445a35d-ab3c-4b04-9e47-41eedd31ef3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:04.917467 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9445a35d-ab3c-4b04-9e47-41eedd31ef3e None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:04.917752 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9445a35d-ab3c-4b04-9e47-41eedd31ef3e None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:04.918030 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9445a35d-ab3c-4b04-9e47-41eedd31ef3e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:04.925903 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9445a35d-ab3c-4b04-9e47-41eedd31ef3e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:04.927396 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9445a35d-ab3c-4b04-9e47-41eedd31ef3e None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:04.942747 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-9445a35d-ab3c-4b04-9e47-41eedd31ef3e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 10 12:45:04.942747 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:04.942747 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:04.942747 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:04.942747 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:04.942747 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:04.942747 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:04.942747 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:04.942747 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:04.942747 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:04.942747 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:04.942747 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:04.942747 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 10 12:45:04.942747 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 10 12:45:04.942747 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 10 12:45:04.942747 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 10 12:45:04.943766 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:04.943766 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:04.943766 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 10 12:45:04.943766 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 10 12:45:04.943766 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 10 12:45:04.943766 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 10 12:45:04.943766 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:04.943766 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:04.943766 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:04.943766 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:04.943766 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:04.943766 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:04.943766 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:04.943766 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:04.943766 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:04.943766 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:04.943766 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:04.943766 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:04.944761 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:04.944761 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:04.944761 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 10 12:45:04.944761 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 10 12:45:04.944761 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 10 12:45:04.944761 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 10 12:45:04.944761 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 10 12:45:04.944761 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 10 12:45:04.944761 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 10 12:45:04.944761 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:04.944761 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 296/578] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Jun 10 12:45:04 2022] GET /identity/v3/users/cinder => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:04.945368 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9445a35d-ab3c-4b04-9e47-41eedd31ef3e None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:04.976425 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0245fb2-0955-4c45-a9e0-01c68c2f54f6 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:05.005442 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:05.017307 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42c6432-9bf9-48b9-948e-1be6b9b77bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:05.018112 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42c6432-9bf9-48b9-948e-1be6b9b77bb1 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:05.018398 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42c6432-9bf9-48b9-948e-1be6b9b77bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:05.021967 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42c6432-9bf9-48b9-948e-1be6b9b77bb1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:05.028038 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42c6432-9bf9-48b9-948e-1be6b9b77bb1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:05.029977 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42c6432-9bf9-48b9-948e-1be6b9b77bb1 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:05.049016 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-a42c6432-9bf9-48b9-948e-1be6b9b77bb1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 10 12:45:05.049016 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:05.049016 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:05.049016 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:05.049016 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:05.049016 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:05.049016 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:05.049016 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:05.049016 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:05.049016 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:05.049016 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:05.049016 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:05.049016 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 10 12:45:05.049016 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 10 12:45:05.049016 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 10 12:45:05.049924 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 10 12:45:05.049924 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:05.049924 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:05.049924 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:05.049924 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:05.049924 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:05.049924 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:05.049924 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:05.049924 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:05.049924 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:05.049924 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:05.049924 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:05.049924 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:05.049924 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:05.049924 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:05.049924 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:05.049924 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:05.050810 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 10 12:45:05.050810 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 10 12:45:05.050810 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 10 12:45:05.050810 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 10 12:45:05.050810 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 10 12:45:05.050810 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 10 12:45:05.050810 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 10 12:45:05.050810 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:05.051994 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 283/579] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Jun 10 12:45:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:05.052895 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42c6432-9bf9-48b9-948e-1be6b9b77bb1 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:05.058701 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9445a35d-ab3c-4b04-9e47-41eedd31ef3e None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:05.069000 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:05.097559 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cea856-28d8-4fe8-a7ab-0328fcb53c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:05.098839 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cea856-28d8-4fe8-a7ab-0328fcb53c1b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:05.099130 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cea856-28d8-4fe8-a7ab-0328fcb53c1b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:05.099408 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cea856-28d8-4fe8-a7ab-0328fcb53c1b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:05.110697 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cea856-28d8-4fe8-a7ab-0328fcb53c1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:05.112212 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cea856-28d8-4fe8-a7ab-0328fcb53c1b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:05.129976 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-14cea856-28d8-4fe8-a7ab-0328fcb53c1b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:05.129976 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:05.129976 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:05.129976 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:05.129976 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:05.129976 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:05.129976 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:05.129976 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:05.129976 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:05.129976 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:05.129976 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:05.129976 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:05.129976 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 10 12:45:05.129976 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 10 12:45:05.129976 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 10 12:45:05.130907 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 10 12:45:05.130907 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:05.130907 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:05.130907 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:05.130907 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:05.130907 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:05.130907 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:05.130907 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:05.130907 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:05.130907 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:05.130907 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:05.130907 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:05.130907 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:05.130907 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:05.130907 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:05.130907 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:05.130907 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:05.131801 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 10 12:45:05.131801 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 10 12:45:05.131801 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:05.131801 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:05.131801 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 297/580] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:45:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:05.132111 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cea856-28d8-4fe8-a7ab-0328fcb53c1b None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:05.163623 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42c6432-9bf9-48b9-948e-1be6b9b77bb1 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:05.199500 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d135c5b0-46e1-4861-ac44-5b0e03deb6ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:05.200767 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d135c5b0-46e1-4861-ac44-5b0e03deb6ac None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:05.201225 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d135c5b0-46e1-4861-ac44-5b0e03deb6ac None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:05.201672 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d135c5b0-46e1-4861-ac44-5b0e03deb6ac None admin] PATH_INFO: `/v3/roles/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:05.213134 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d135c5b0-46e1-4861-ac44-5b0e03deb6ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:45:05.221429 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-d135c5b0-46e1-4861-ac44-5b0e03deb6ac None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 10 12:45:05.221429 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:05.221429 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:05.221429 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:05.221429 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:05.221429 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:05.221429 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:05.221429 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:05.221429 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:05.221429 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:05.221429 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:05.221429 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:05.221429 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:45:05.221429 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:45:05.221429 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:45:05.221429 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:45:05.222459 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:45:05.222459 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:45:05.222459 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:45:05.222459 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:45:05.222459 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:05.222459 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:05.222459 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:05.222459 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:05.222459 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:05.222459 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:05.222459 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:05.222459 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:05.222459 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:05.222459 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:05.222459 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:05.222459 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:05.223871 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:05.223871 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:05.223871 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:05.223871 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:05.223871 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:45:05.223871 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:45:05.223871 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:45:05.223871 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:45:05.223871 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:45:05.223871 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:45:05.223871 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 10 12:45:05.223871 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:05.223871 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 284/581] 10.222.0.27 () {56 vars in 1102 bytes} [Fri Jun 10 12:45:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:05.223871 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d135c5b0-46e1-4861-ac44-5b0e03deb6ac None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:05.240628 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cea856-28d8-4fe8-a7ab-0328fcb53c1b None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:05.243628 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:05.280041 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2a12b6-6bf6-4b51-a6e5-686f9015722c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:05.280720 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2a12b6-6bf6-4b51-a6e5-686f9015722c None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:05.281003 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2a12b6-6bf6-4b51-a6e5-686f9015722c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:05.281275 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2a12b6-6bf6-4b51-a6e5-686f9015722c None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:05.282121 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2a12b6-6bf6-4b51-a6e5-686f9015722c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:05.282431 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2a12b6-6bf6-4b51-a6e5-686f9015722c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:05.283890 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2a12b6-6bf6-4b51-a6e5-686f9015722c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:05.303315 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 298/582] 10.222.0.27 () {56 vars in 1137 bytes} [Fri Jun 10 12:45:05 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:05.304178 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2a12b6-6bf6-4b51-a6e5-686f9015722c None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:05.314130 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:05.340175 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d135c5b0-46e1-4861-ac44-5b0e03deb6ac None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:05.375685 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1ad57e-8f42-4ba4-84ee-a739ac7cfd07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:05.376392 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1ad57e-8f42-4ba4-84ee-a739ac7cfd07 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:05.376676 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1ad57e-8f42-4ba4-84ee-a739ac7cfd07 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:05.376955 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1ad57e-8f42-4ba4-84ee-a739ac7cfd07 None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:05.377766 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1ad57e-8f42-4ba4-84ee-a739ac7cfd07 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:05.378054 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1ad57e-8f42-4ba4-84ee-a739ac7cfd07 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:05.379472 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1ad57e-8f42-4ba4-84ee-a739ac7cfd07 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:05.392394 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 285/583] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Jun 10 12:45:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:05.393183 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1ad57e-8f42-4ba4-84ee-a739ac7cfd07 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:05.417793 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2a12b6-6bf6-4b51-a6e5-686f9015722c None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:05.454774 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e01a33-35df-4fad-b13a-feae7b414c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:05.455594 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e01a33-35df-4fad-b13a-feae7b414c5d None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:05.456188 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e01a33-35df-4fad-b13a-feae7b414c5d None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:05.456476 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e01a33-35df-4fad-b13a-feae7b414c5d None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:05.457343 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e01a33-35df-4fad-b13a-feae7b414c5d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:05.457654 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e01a33-35df-4fad-b13a-feae7b414c5d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:05.459123 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e01a33-35df-4fad-b13a-feae7b414c5d None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:05.471052 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 299/584] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Jun 10 12:45:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:05.471678 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e01a33-35df-4fad-b13a-feae7b414c5d None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:05.496453 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:05.499187 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1ad57e-8f42-4ba4-84ee-a739ac7cfd07 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:05.536207 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c97a1e8-f940-452c-87e0-c0ce15038322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:05.537418 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c97a1e8-f940-452c-87e0-c0ce15038322 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:05.537878 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c97a1e8-f940-452c-87e0-c0ce15038322 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:05.538327 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c97a1e8-f940-452c-87e0-c0ce15038322 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:05.539291 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c97a1e8-f940-452c-87e0-c0ce15038322 None admin] RBAC: Adding query filter params (name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:05.539772 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c97a1e8-f940-452c-87e0-c0ce15038322 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:05.541341 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c97a1e8-f940-452c-87e0-c0ce15038322 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:05.542980 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:05.555106 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 286/585] 10.222.0.27 () {56 vars in 1103 bytes} [Fri Jun 10 12:45:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:05.555106 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c97a1e8-f940-452c-87e0-c0ce15038322 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:05.582420 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e01a33-35df-4fad-b13a-feae7b414c5d None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:05.620689 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bee6deb-968e-4c24-80bb-92c5f17bc441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:05.621479 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bee6deb-968e-4c24-80bb-92c5f17bc441 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:05.621760 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bee6deb-968e-4c24-80bb-92c5f17bc441 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:05.622034 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bee6deb-968e-4c24-80bb-92c5f17bc441 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:05.634060 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bee6deb-968e-4c24-80bb-92c5f17bc441 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:05.635427 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bee6deb-968e-4c24-80bb-92c5f17bc441 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:05.652153 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-2bee6deb-968e-4c24-80bb-92c5f17bc441 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:05.652153 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:05.652153 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:05.652153 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:05.652153 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:05.652153 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:05.652153 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:05.652153 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:05.652153 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:05.652153 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:05.652153 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:05.652153 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:05.652153 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 10 12:45:05.652153 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 10 12:45:05.652153 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 10 12:45:05.653054 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 10 12:45:05.653054 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:05.653054 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:05.653054 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:05.653054 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:05.653054 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:05.653054 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:05.653054 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:05.653054 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:05.653054 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:05.653054 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:05.653054 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:05.653054 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:05.653054 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:05.653054 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:05.653054 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:05.653054 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:05.653942 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 10 12:45:05.653942 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 10 12:45:05.653942 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:05.653942 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:05.653942 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 300/586] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:45:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:05.654227 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bee6deb-968e-4c24-80bb-92c5f17bc441 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:05.660902 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c97a1e8-f940-452c-87e0-c0ce15038322 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:05.694764 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9568be-b14f-44b8-b0af-4f8a4153b093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:05.696223 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9568be-b14f-44b8-b0af-4f8a4153b093 None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:05.696510 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9568be-b14f-44b8-b0af-4f8a4153b093 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:05.696785 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9568be-b14f-44b8-b0af-4f8a4153b093 None admin] PATH_INFO: `/v3/projects/73ea5b23a2d34544ab493f4c5e4034b8/users/6d3eb01a98274050b930616b98998c06/roles/ea7e8f74523b4322857fb3f4307d171e` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:05.710902 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9568be-b14f-44b8-b0af-4f8a4153b093 None admin] RBAC: Authorizing `identity:create_grant(project_id=73ea5b23a2d34544ab493f4c5e4034b8, user_id=6d3eb01a98274050b930616b98998c06, role_id=ea7e8f74523b4322857fb3f4307d171e)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:05.712392 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9568be-b14f-44b8-b0af-4f8a4153b093 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:05.733150 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:05.756520 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bee6deb-968e-4c24-80bb-92c5f17bc441 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:05.788093 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4241df04-bc03-4eaf-8cca-7f67cfc42585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:05.788865 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4241df04-bc03-4eaf-8cca-7f67cfc42585 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:05.789187 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4241df04-bc03-4eaf-8cca-7f67cfc42585 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:05.789494 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4241df04-bc03-4eaf-8cca-7f67cfc42585 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:05.798887 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 287/587] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Jun 10 12:45:05 2022] PUT /identity/v3/projects/73ea5b23a2d34544ab493f4c5e4034b8/users/6d3eb01a98274050b930616b98998c06/roles/ea7e8f74523b4322857fb3f4307d171e => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 12:45:05.799698 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9568be-b14f-44b8-b0af-4f8a4153b093 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:05.800981 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4241df04-bc03-4eaf-8cca-7f67cfc42585 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:05.802441 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4241df04-bc03-4eaf-8cca-7f67cfc42585 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:05.821016 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-4241df04-bc03-4eaf-8cca-7f67cfc42585 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 10 12:45:05.821016 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:05.821016 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:05.821016 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:05.821016 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:05.821016 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:05.821016 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:05.821016 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:05.821016 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:05.821016 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:05.821016 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:05.821016 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:05.821016 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 10 12:45:05.821016 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 10 12:45:05.821016 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 10 12:45:05.821016 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 10 12:45:05.822067 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:05.822067 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:05.822067 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 10 12:45:05.822067 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 10 12:45:05.822067 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 10 12:45:05.822067 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 10 12:45:05.822067 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:05.822067 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:05.822067 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:05.822067 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:05.822067 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:05.822067 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:05.822067 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:05.822067 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:05.822067 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:05.822067 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:05.822067 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:05.822067 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:05.823015 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:05.823015 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:05.823015 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 10 12:45:05.823015 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 10 12:45:05.823015 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 10 12:45:05.823015 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 10 12:45:05.823015 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 10 12:45:05.823015 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 10 12:45:05.823015 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 10 12:45:05.823015 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:05.830016 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 301/588] 10.222.0.27 () {56 vars in 1093 bytes} [Fri Jun 10 12:45:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:05.831334 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4241df04-bc03-4eaf-8cca-7f67cfc42585 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:05.870562 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:05.934441 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9568be-b14f-44b8-b0af-4f8a4153b093 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:05.973775 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1408534-9612-4931-a4ed-c95fc930b048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:05.974489 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4241df04-bc03-4eaf-8cca-7f67cfc42585 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:05.974602 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1408534-9612-4931-a4ed-c95fc930b048 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:05.974895 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1408534-9612-4931-a4ed-c95fc930b048 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:05.975173 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1408534-9612-4931-a4ed-c95fc930b048 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:05.987379 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1408534-9612-4931-a4ed-c95fc930b048 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:05.988974 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1408534-9612-4931-a4ed-c95fc930b048 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:06.007914 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-c1408534-9612-4931-a4ed-c95fc930b048 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:06.007914 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:06.007914 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:06.007914 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:06.007914 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:06.007914 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:06.007914 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:06.007914 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:06.007914 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:06.007914 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:06.007914 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:06.007914 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:06.007914 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 10 12:45:06.007914 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 10 12:45:06.007914 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 10 12:45:06.008983 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 10 12:45:06.008983 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:06.008983 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:06.008983 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:06.008983 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:06.008983 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:06.008983 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:06.008983 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:06.008983 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:06.008983 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:06.008983 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:06.008983 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:06.008983 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:06.008983 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:06.008983 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:06.008983 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:06.008983 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:06.010021 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 10 12:45:06.010021 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 10 12:45:06.010021 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:06.010021 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:06.010313 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 288/589] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:45:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:06.011101 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1408534-9612-4931-a4ed-c95fc930b048 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:06.019739 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb615f40-9b6f-4b1e-a782-cec0c971df1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:06.020437 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb615f40-9b6f-4b1e-a782-cec0c971df1c None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:06.020720 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb615f40-9b6f-4b1e-a782-cec0c971df1c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:06.021017 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb615f40-9b6f-4b1e-a782-cec0c971df1c None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:06.021748 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb615f40-9b6f-4b1e-a782-cec0c971df1c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:06.022053 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb615f40-9b6f-4b1e-a782-cec0c971df1c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:06.023426 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb615f40-9b6f-4b1e-a782-cec0c971df1c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:06.025724 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:06.041284 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 302/590] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Jun 10 12:45:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:06.045822 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb615f40-9b6f-4b1e-a782-cec0c971df1c None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:06.129482 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:06.135332 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1408534-9612-4931-a4ed-c95fc930b048 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:06.170956 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb615f40-9b6f-4b1e-a782-cec0c971df1c None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:06.179678 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9da4c86-a64f-40e6-a702-128a51facb9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:06.181753 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9da4c86-a64f-40e6-a702-128a51facb9f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:06.182245 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9da4c86-a64f-40e6-a702-128a51facb9f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:06.183165 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9da4c86-a64f-40e6-a702-128a51facb9f None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:06.184740 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9da4c86-a64f-40e6-a702-128a51facb9f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:06.185414 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9da4c86-a64f-40e6-a702-128a51facb9f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:06.187388 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9da4c86-a64f-40e6-a702-128a51facb9f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:06.209972 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 289/591] 10.222.0.27 () {56 vars in 1133 bytes} [Fri Jun 10 12:45:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:06.211005 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9da4c86-a64f-40e6-a702-128a51facb9f None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:06.213756 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35a559f-58e5-439b-8d07-4c567fb7aed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:06.214760 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35a559f-58e5-439b-8d07-4c567fb7aed0 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:06.215251 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35a559f-58e5-439b-8d07-4c567fb7aed0 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:06.215732 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35a559f-58e5-439b-8d07-4c567fb7aed0 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:06.216767 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35a559f-58e5-439b-8d07-4c567fb7aed0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:06.217295 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35a559f-58e5-439b-8d07-4c567fb7aed0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:06.218885 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35a559f-58e5-439b-8d07-4c567fb7aed0 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:06.232454 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 303/592] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Jun 10 12:45:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:06.233649 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35a559f-58e5-439b-8d07-4c567fb7aed0 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:06.292747 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:06.322143 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9da4c86-a64f-40e6-a702-128a51facb9f None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:06.341843 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35a559f-58e5-439b-8d07-4c567fb7aed0 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:06.360322 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcdab51-ae2f-4ff4-9412-6a47a478a536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:06.360945 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcdab51-ae2f-4ff4-9412-6a47a478a536 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:06.361230 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcdab51-ae2f-4ff4-9412-6a47a478a536 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:06.361507 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcdab51-ae2f-4ff4-9412-6a47a478a536 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:06.368860 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcdab51-ae2f-4ff4-9412-6a47a478a536 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:06.370293 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcdab51-ae2f-4ff4-9412-6a47a478a536 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:06.382368 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c05a365-03f4-40d6-9575-ed9b98d42ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:06.383498 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c05a365-03f4-40d6-9575-ed9b98d42ac0 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:06.384010 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c05a365-03f4-40d6-9575-ed9b98d42ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:06.384512 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c05a365-03f4-40d6-9575-ed9b98d42ac0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:06.384786 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-6fcdab51-ae2f-4ff4-9412-6a47a478a536 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 10 12:45:06.384786 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:06.384786 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:06.384786 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:06.384786 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:06.384786 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:06.384786 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:06.384786 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:06.384786 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:06.384786 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:06.384786 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:06.384786 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:06.384786 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 10 12:45:06.384786 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 10 12:45:06.384786 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 10 12:45:06.385655 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 10 12:45:06.385655 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:06.385655 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:06.385655 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:06.385655 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:06.385655 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:06.385655 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:06.385655 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:06.385655 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:06.385655 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:06.385655 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:06.385655 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:06.385655 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:06.385655 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:06.385655 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:06.385655 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:06.385655 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:06.386569 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 10 12:45:06.386569 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 10 12:45:06.386569 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 10 12:45:06.386569 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 10 12:45:06.386569 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 10 12:45:06.386569 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 10 12:45:06.386569 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 10 12:45:06.386569 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:06.386569 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 290/593] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Jun 10 12:45:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:06.387070 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcdab51-ae2f-4ff4-9412-6a47a478a536 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:06.399715 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c05a365-03f4-40d6-9575-ed9b98d42ac0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:06.401594 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c05a365-03f4-40d6-9575-ed9b98d42ac0 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:06.407224 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:06.421580 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-3c05a365-03f4-40d6-9575-ed9b98d42ac0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:06.421580 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:06.421580 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:06.421580 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:06.421580 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:06.421580 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:06.421580 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:06.421580 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:06.421580 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:06.421580 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:06.421580 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:06.421580 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:06.421580 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 10 12:45:06.421580 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 10 12:45:06.421580 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 10 12:45:06.422502 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 10 12:45:06.422502 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:06.422502 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:06.422502 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:06.422502 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:06.422502 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:06.422502 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:06.422502 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:06.422502 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:06.422502 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:06.422502 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:06.422502 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:06.422502 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:06.422502 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:06.422502 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:06.422502 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:06.422502 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:06.423409 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 10 12:45:06.423409 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 10 12:45:06.423409 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:06.423409 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:06.425623 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 304/594] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:45:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:06.426605 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c05a365-03f4-40d6-9575-ed9b98d42ac0 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:06.494769 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcdab51-ae2f-4ff4-9412-6a47a478a536 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:06.532329 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25509a68-a6b8-4fe6-99de-ef82ee668c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:06.533252 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25509a68-a6b8-4fe6-99de-ef82ee668c79 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:06.533453 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25509a68-a6b8-4fe6-99de-ef82ee668c79 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:06.533723 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25509a68-a6b8-4fe6-99de-ef82ee668c79 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:06.535999 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c05a365-03f4-40d6-9575-ed9b98d42ac0 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:06.542737 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25509a68-a6b8-4fe6-99de-ef82ee668c79 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:06.544208 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25509a68-a6b8-4fe6-99de-ef82ee668c79 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:06.545327 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:06.562022 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-25509a68-a6b8-4fe6-99de-ef82ee668c79 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 10 12:45:06.562022 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:06.562022 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:06.562022 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:06.562022 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:06.562022 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:06.562022 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:06.562022 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:06.562022 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:06.562022 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:06.562022 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:06.562022 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:06.562022 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 10 12:45:06.562022 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 10 12:45:06.562022 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 10 12:45:06.562022 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 10 12:45:06.563078 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:06.563078 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:06.563078 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 10 12:45:06.563078 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 10 12:45:06.563078 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 10 12:45:06.563078 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 10 12:45:06.563078 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:06.563078 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:06.563078 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:06.563078 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:06.563078 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:06.563078 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:06.563078 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:06.563078 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:06.563078 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:06.563078 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:06.563078 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:06.563078 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:06.564052 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:06.564052 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:06.564052 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 10 12:45:06.564052 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 10 12:45:06.564052 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 10 12:45:06.564052 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 10 12:45:06.564052 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 10 12:45:06.564052 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 10 12:45:06.564052 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 10 12:45:06.564052 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:06.571342 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 291/595] 10.222.0.27 () {56 vars in 1102 bytes} [Fri Jun 10 12:45:06 2022] GET /identity/v3/users/neutron => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:06.573480 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25509a68-a6b8-4fe6-99de-ef82ee668c79 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:06.574241 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0f9be4-b9e4-4544-8ff2-3c8bd7c5621f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:06.574999 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0f9be4-b9e4-4544-8ff2-3c8bd7c5621f None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:06.575356 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0f9be4-b9e4-4544-8ff2-3c8bd7c5621f None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:06.575706 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0f9be4-b9e4-4544-8ff2-3c8bd7c5621f None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:06.576613 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0f9be4-b9e4-4544-8ff2-3c8bd7c5621f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:06.577007 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0f9be4-b9e4-4544-8ff2-3c8bd7c5621f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:06.579179 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0f9be4-b9e4-4544-8ff2-3c8bd7c5621f None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:06.593295 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 305/596] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Jun 10 12:45:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:06.594318 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0f9be4-b9e4-4544-8ff2-3c8bd7c5621f None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:06.665269 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:06.689262 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25509a68-a6b8-4fe6-99de-ef82ee668c79 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:06.704456 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0f9be4-b9e4-4544-8ff2-3c8bd7c5621f None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:06.727795 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f7188f-72d8-41d7-87f8-9e523c62d0da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:06.728696 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f7188f-72d8-41d7-87f8-9e523c62d0da None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:06.729074 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f7188f-72d8-41d7-87f8-9e523c62d0da None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:06.729445 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f7188f-72d8-41d7-87f8-9e523c62d0da None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:06.730263 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f7188f-72d8-41d7-87f8-9e523c62d0da None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:06.730665 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f7188f-72d8-41d7-87f8-9e523c62d0da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:06.732265 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f7188f-72d8-41d7-87f8-9e523c62d0da None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:06.743668 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a28ef6-b78a-4216-812b-d299ee556b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:06.744582 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a28ef6-b78a-4216-812b-d299ee556b43 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:06.745036 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 292/597] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Jun 10 12:45:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:06.745313 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a28ef6-b78a-4216-812b-d299ee556b43 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:06.745766 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a28ef6-b78a-4216-812b-d299ee556b43 None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:06.746021 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f7188f-72d8-41d7-87f8-9e523c62d0da None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:06.747082 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a28ef6-b78a-4216-812b-d299ee556b43 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:06.747582 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a28ef6-b78a-4216-812b-d299ee556b43 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:06.749689 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a28ef6-b78a-4216-812b-d299ee556b43 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:06.769271 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 306/598] 10.222.0.27 () {56 vars in 1139 bytes} [Fri Jun 10 12:45:06 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:06.770518 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a28ef6-b78a-4216-812b-d299ee556b43 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:06.797814 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:06.855917 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f7188f-72d8-41d7-87f8-9e523c62d0da None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:06.877942 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a28ef6-b78a-4216-812b-d299ee556b43 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:06.901739 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff13769c-8163-4513-84a2-ed59405a972e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:06.902925 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff13769c-8163-4513-84a2-ed59405a972e None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:06.903361 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff13769c-8163-4513-84a2-ed59405a972e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:06.903777 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff13769c-8163-4513-84a2-ed59405a972e None admin] PATH_INFO: `/v3/projects/73ea5b23a2d34544ab493f4c5e4034b8/users/c6cc95a563b941ac92ab283aa0d5a5d5/roles/ea7e8f74523b4322857fb3f4307d171e` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:06.908771 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:06.919273 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cc6547-6bf0-4fec-8c20-e0c869733172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:06.920232 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cc6547-6bf0-4fec-8c20-e0c869733172 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:06.920575 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cc6547-6bf0-4fec-8c20-e0c869733172 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:06.920918 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cc6547-6bf0-4fec-8c20-e0c869733172 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:06.924407 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff13769c-8163-4513-84a2-ed59405a972e None admin] RBAC: Authorizing `identity:create_grant(project_id=73ea5b23a2d34544ab493f4c5e4034b8, user_id=c6cc95a563b941ac92ab283aa0d5a5d5, role_id=ea7e8f74523b4322857fb3f4307d171e)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:06.925102 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff13769c-8163-4513-84a2-ed59405a972e None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:06.929559 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cc6547-6bf0-4fec-8c20-e0c869733172 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:06.931167 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cc6547-6bf0-4fec-8c20-e0c869733172 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:06.947380 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-a2cc6547-6bf0-4fec-8c20-e0c869733172 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 10 12:45:06.947380 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:06.947380 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:06.947380 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:06.947380 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:06.947380 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:06.947380 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:06.947380 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:06.947380 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:06.947380 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:06.947380 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:06.947380 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:06.947380 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 10 12:45:06.947380 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 10 12:45:06.947380 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 10 12:45:06.948393 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 10 12:45:06.948393 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:06.948393 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:06.948393 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:06.948393 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:06.948393 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:06.948393 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:06.948393 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:06.948393 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:06.948393 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:06.948393 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:06.948393 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:06.948393 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:06.948393 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:06.948393 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:06.948393 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:06.948393 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:06.949382 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 10 12:45:06.949382 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 10 12:45:06.949382 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 10 12:45:06.949382 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 10 12:45:06.949382 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 10 12:45:06.949382 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 10 12:45:06.949382 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 10 12:45:06.949382 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:06.949382 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 307/599] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Jun 10 12:45:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:06.949877 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cc6547-6bf0-4fec-8c20-e0c869733172 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:06.991327 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 293/600] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Jun 10 12:45:06 2022] PUT /identity/v3/projects/73ea5b23a2d34544ab493f4c5e4034b8/users/c6cc95a563b941ac92ab283aa0d5a5d5/roles/ea7e8f74523b4322857fb3f4307d171e => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 12:45:06.992597 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff13769c-8163-4513-84a2-ed59405a972e None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:07.053660 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:07.078560 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cc6547-6bf0-4fec-8c20-e0c869733172 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:07.116799 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fadbf9-425b-455f-9770-c9afbbe5b0bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:07.118320 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fadbf9-425b-455f-9770-c9afbbe5b0bf None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:07.118711 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fadbf9-425b-455f-9770-c9afbbe5b0bf None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:07.119101 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fadbf9-425b-455f-9770-c9afbbe5b0bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:07.131074 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff13769c-8163-4513-84a2-ed59405a972e None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:07.133999 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fadbf9-425b-455f-9770-c9afbbe5b0bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:07.135584 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fadbf9-425b-455f-9770-c9afbbe5b0bf None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:07.154960 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-57fadbf9-425b-455f-9770-c9afbbe5b0bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:07.154960 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:07.154960 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:07.154960 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:07.154960 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:07.154960 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:07.154960 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:07.154960 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:07.154960 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:07.154960 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:07.154960 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:07.154960 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:07.154960 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 10 12:45:07.154960 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 10 12:45:07.154960 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 10 12:45:07.156340 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 10 12:45:07.156340 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:07.156340 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:07.156340 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:07.156340 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:07.156340 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:07.156340 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:07.156340 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:07.156340 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:07.156340 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:07.156340 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:07.156340 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:07.156340 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:07.156340 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:07.156340 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:07.156340 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:07.156340 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:07.158228 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 10 12:45:07.158228 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 10 12:45:07.158228 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:07.158228 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:07.158228 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 308/601] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:45:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:07.163227 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d601d1-4c2f-4304-9ad8-53d4125f7900 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:07.163227 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d601d1-4c2f-4304-9ad8-53d4125f7900 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:07.163227 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d601d1-4c2f-4304-9ad8-53d4125f7900 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:07.163227 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 309/602] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:45:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:45:07.164528 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d601d1-4c2f-4304-9ad8-53d4125f7900 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:07.174916 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddee8182-743e-48e3-aad6-b0d4756e610b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:07.175892 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddee8182-743e-48e3-aad6-b0d4756e610b None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:07.176354 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddee8182-743e-48e3-aad6-b0d4756e610b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:07.176794 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddee8182-743e-48e3-aad6-b0d4756e610b None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:07.177783 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddee8182-743e-48e3-aad6-b0d4756e610b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:07.178265 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddee8182-743e-48e3-aad6-b0d4756e610b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:07.180491 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddee8182-743e-48e3-aad6-b0d4756e610b None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:07.193585 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 294/603] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Jun 10 12:45:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:07.194852 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254e5c1f-6a7e-4814-80bf-b932e6d8a210 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:07.195140 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254e5c1f-6a7e-4814-80bf-b932e6d8a210 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:07.196778 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254e5c1f-6a7e-4814-80bf-b932e6d8a210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:07.214214 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-254e5c1f-6a7e-4814-80bf-b932e6d8a210 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:45:07.215430 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-254e5c1f-6a7e-4814-80bf-b932e6d8a210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:45:07.271273 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:07.290585 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d601d1-4c2f-4304-9ad8-53d4125f7900 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:07.303449 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 295/604] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:45:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 10 12:45:07.307971 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254e5c1f-6a7e-4814-80bf-b932e6d8a210 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:07.356159 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da887a97-2a73-413c-b7b6-1b33ebee7c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:07.357118 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da887a97-2a73-413c-b7b6-1b33ebee7c10 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:07.357410 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da887a97-2a73-413c-b7b6-1b33ebee7c10 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:07.357694 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da887a97-2a73-413c-b7b6-1b33ebee7c10 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:07.358426 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da887a97-2a73-413c-b7b6-1b33ebee7c10 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:07.358732 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da887a97-2a73-413c-b7b6-1b33ebee7c10 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:07.360556 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da887a97-2a73-413c-b7b6-1b33ebee7c10 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:07.370996 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:07.378515 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 310/605] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Jun 10 12:45:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:07.380109 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da887a97-2a73-413c-b7b6-1b33ebee7c10 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:07.417114 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254e5c1f-6a7e-4814-80bf-b932e6d8a210 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:07.454851 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d11b34-c8f9-45fc-b5a9-7b2ca333fa94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:07.455786 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d11b34-c8f9-45fc-b5a9-7b2ca333fa94 None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:07.456181 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d11b34-c8f9-45fc-b5a9-7b2ca333fa94 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:07.456541 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d11b34-c8f9-45fc-b5a9-7b2ca333fa94 None admin] PATH_INFO: `/v3/projects/73ea5b23a2d34544ab493f4c5e4034b8/users/8bd6caa461e84100a3d5fe5af5d80f3e/roles/ea7e8f74523b4322857fb3f4307d171e` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:07.474045 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d11b34-c8f9-45fc-b5a9-7b2ca333fa94 None admin] RBAC: Authorizing `identity:create_grant(project_id=73ea5b23a2d34544ab493f4c5e4034b8, user_id=8bd6caa461e84100a3d5fe5af5d80f3e, role_id=ea7e8f74523b4322857fb3f4307d171e)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:07.475696 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d11b34-c8f9-45fc-b5a9-7b2ca333fa94 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:07.509148 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da887a97-2a73-413c-b7b6-1b33ebee7c10 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:07.545262 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c8a8fa-f32d-421a-98cb-d214e4a732ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:07.546125 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c8a8fa-f32d-421a-98cb-d214e4a732ed None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:07.546407 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c8a8fa-f32d-421a-98cb-d214e4a732ed None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:07.546683 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c8a8fa-f32d-421a-98cb-d214e4a732ed None admin] PATH_INFO: `/v3/roles/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:07.559788 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 296/606] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Jun 10 12:45:07 2022] PUT /identity/v3/projects/73ea5b23a2d34544ab493f4c5e4034b8/users/8bd6caa461e84100a3d5fe5af5d80f3e/roles/ea7e8f74523b4322857fb3f4307d171e => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 12:45:07.561684 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d11b34-c8f9-45fc-b5a9-7b2ca333fa94 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:07.568216 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c8a8fa-f32d-421a-98cb-d214e4a732ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:45:07.576799 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-d2c8a8fa-f32d-421a-98cb-d214e4a732ed None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 10 12:45:07.576799 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:07.576799 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:07.576799 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:07.576799 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:07.576799 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:07.576799 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:07.576799 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:07.576799 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:07.576799 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:07.576799 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:07.576799 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:07.576799 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:45:07.576799 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:45:07.576799 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:45:07.576799 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:45:07.577892 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:45:07.577892 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:45:07.577892 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:45:07.577892 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:45:07.577892 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:07.577892 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:07.577892 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:07.577892 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:07.577892 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:07.577892 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:07.577892 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:07.577892 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:07.577892 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:07.577892 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:07.577892 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:07.577892 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:07.578734 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:07.578734 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:07.578734 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:07.578734 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:07.578734 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:45:07.578734 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:45:07.578734 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:45:07.578734 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:45:07.578734 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:45:07.578734 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:45:07.578734 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 10 12:45:07.578734 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:07.583394 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 311/607] 10.222.0.27 () {56 vars in 1102 bytes} [Fri Jun 10 12:45:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:07.584367 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c8a8fa-f32d-421a-98cb-d214e4a732ed None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:07.619929 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:07.718401 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:07.735015 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d11b34-c8f9-45fc-b5a9-7b2ca333fa94 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:07.794752 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af7e545-1af6-4c5d-9311-671fb3f252f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:07.795565 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af7e545-1af6-4c5d-9311-671fb3f252f8 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:07.795851 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af7e545-1af6-4c5d-9311-671fb3f252f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:07.796541 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c8a8fa-f32d-421a-98cb-d214e4a732ed None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:07.796654 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af7e545-1af6-4c5d-9311-671fb3f252f8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:07.806086 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af7e545-1af6-4c5d-9311-671fb3f252f8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:07.807569 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af7e545-1af6-4c5d-9311-671fb3f252f8 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:07.829388 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-5af7e545-1af6-4c5d-9311-671fb3f252f8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 10 12:45:07.829388 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:07.829388 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:07.829388 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:07.829388 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:07.829388 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:07.829388 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:07.829388 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:07.829388 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:07.829388 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:07.829388 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:07.829388 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:07.829388 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 10 12:45:07.829388 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 10 12:45:07.829388 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 10 12:45:07.830329 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 10 12:45:07.830329 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:07.830329 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:07.830329 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:07.830329 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:07.830329 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:07.830329 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:07.830329 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:07.830329 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:07.830329 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:07.830329 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:07.830329 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:07.830329 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:07.830329 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:07.830329 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:07.830329 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:07.830329 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:07.831967 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 10 12:45:07.831967 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 10 12:45:07.831967 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 10 12:45:07.831967 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 10 12:45:07.831967 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 10 12:45:07.831967 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 10 12:45:07.831967 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 10 12:45:07.831967 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:07.831967 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 297/608] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Jun 10 12:45:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:07.837210 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af7e545-1af6-4c5d-9311-671fb3f252f8 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:07.846974 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8b4a3c-2709-4c86-bdac-7f8881c1814e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:07.848055 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8b4a3c-2709-4c86-bdac-7f8881c1814e None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:07.848609 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8b4a3c-2709-4c86-bdac-7f8881c1814e None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:07.850092 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8b4a3c-2709-4c86-bdac-7f8881c1814e None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:07.851160 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8b4a3c-2709-4c86-bdac-7f8881c1814e None admin] RBAC: Adding query filter params (name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:07.852871 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8b4a3c-2709-4c86-bdac-7f8881c1814e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:07.854652 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8b4a3c-2709-4c86-bdac-7f8881c1814e None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:07.864697 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 312/609] 10.222.0.27 () {56 vars in 1103 bytes} [Fri Jun 10 12:45:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:07.877197 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8b4a3c-2709-4c86-bdac-7f8881c1814e None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:07.936750 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:07.953210 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af7e545-1af6-4c5d-9311-671fb3f252f8 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:07.989085 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f45b6f-a38f-4d76-9c4f-7342e8ab4c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:07.990076 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f45b6f-a38f-4d76-9c4f-7342e8ab4c0c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:07.990533 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f45b6f-a38f-4d76-9c4f-7342e8ab4c0c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:07.990958 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f45b6f-a38f-4d76-9c4f-7342e8ab4c0c None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:07.991953 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f45b6f-a38f-4d76-9c4f-7342e8ab4c0c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:07.992864 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f45b6f-a38f-4d76-9c4f-7342e8ab4c0c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:07.994458 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f45b6f-a38f-4d76-9c4f-7342e8ab4c0c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:08.007732 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8b4a3c-2709-4c86-bdac-7f8881c1814e None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:08.009312 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 298/610] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Jun 10 12:45:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:08.018506 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f45b6f-a38f-4d76-9c4f-7342e8ab4c0c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:08.048548 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:08.054432 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eec4ca3-2e37-4a78-a7c9-046d06dedd1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:08.056252 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eec4ca3-2e37-4a78-a7c9-046d06dedd1b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:08.056551 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eec4ca3-2e37-4a78-a7c9-046d06dedd1b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:08.056840 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eec4ca3-2e37-4a78-a7c9-046d06dedd1b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:08.089820 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eec4ca3-2e37-4a78-a7c9-046d06dedd1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:08.091365 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eec4ca3-2e37-4a78-a7c9-046d06dedd1b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:08.110919 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-6eec4ca3-2e37-4a78-a7c9-046d06dedd1b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:08.110919 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:08.110919 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:08.110919 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:08.110919 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:08.110919 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:08.110919 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:08.110919 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:08.110919 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:08.110919 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:08.110919 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:08.110919 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:08.110919 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 10 12:45:08.110919 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 10 12:45:08.110919 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 10 12:45:08.111944 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 10 12:45:08.111944 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:08.111944 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:08.111944 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:08.111944 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:08.111944 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:08.111944 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:08.111944 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:08.111944 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:08.111944 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:08.111944 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:08.111944 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:08.111944 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:08.111944 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:08.111944 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:08.111944 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:08.111944 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:08.113300 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 10 12:45:08.113300 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 10 12:45:08.113300 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:08.113300 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:08.113300 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 313/611] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:45:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:08.118304 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eec4ca3-2e37-4a78-a7c9-046d06dedd1b None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:08.189820 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f45b6f-a38f-4d76-9c4f-7342e8ab4c0c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:08.221619 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eec4ca3-2e37-4a78-a7c9-046d06dedd1b None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:08.232737 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00bcc99-30d0-4eb6-a531-9aa24c80cca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:08.233955 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00bcc99-30d0-4eb6-a531-9aa24c80cca2 None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:08.234397 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00bcc99-30d0-4eb6-a531-9aa24c80cca2 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:08.234800 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00bcc99-30d0-4eb6-a531-9aa24c80cca2 None admin] PATH_INFO: `/v3/projects/73ea5b23a2d34544ab493f4c5e4034b8/users/bc4415461a974c24ad7105a97c7d915b/roles/ea7e8f74523b4322857fb3f4307d171e` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:08.257202 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00bcc99-30d0-4eb6-a531-9aa24c80cca2 None admin] RBAC: Authorizing `identity:create_grant(project_id=73ea5b23a2d34544ab493f4c5e4034b8, user_id=bc4415461a974c24ad7105a97c7d915b, role_id=ea7e8f74523b4322857fb3f4307d171e)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:08.259193 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00bcc99-30d0-4eb6-a531-9aa24c80cca2 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:08.261644 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608cb123-df9e-40d6-8a9c-ce635c6fb182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:08.262241 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:08.263388 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608cb123-df9e-40d6-8a9c-ce635c6fb182 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:08.263762 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608cb123-df9e-40d6-8a9c-ce635c6fb182 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:08.264012 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608cb123-df9e-40d6-8a9c-ce635c6fb182 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:08.264736 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608cb123-df9e-40d6-8a9c-ce635c6fb182 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:08.265057 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608cb123-df9e-40d6-8a9c-ce635c6fb182 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:08.266412 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608cb123-df9e-40d6-8a9c-ce635c6fb182 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:08.279115 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 314/612] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Jun 10 12:45:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:08.284250 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2804b810-708c-4cc0-bb4c-a59788bb6eba None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:08.284603 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2804b810-708c-4cc0-bb4c-a59788bb6eba None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:08.284833 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2804b810-708c-4cc0-bb4c-a59788bb6eba None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:08.287676 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 315/613] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:45:08 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:45:08.287676 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2804b810-708c-4cc0-bb4c-a59788bb6eba None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:08.317488 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:08.370802 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 299/614] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Jun 10 12:45:08 2022] PUT /identity/v3/projects/73ea5b23a2d34544ab493f4c5e4034b8/users/bc4415461a974c24ad7105a97c7d915b/roles/ea7e8f74523b4322857fb3f4307d171e => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 12:45:08.372897 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b7d358-5802-4ff4-ac2f-7d78ef04d1f7 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:08.372897 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b7d358-5802-4ff4-ac2f-7d78ef04d1f7 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:08.373114 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b7d358-5802-4ff4-ac2f-7d78ef04d1f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:08.410804 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-75b7d358-5802-4ff4-ac2f-7d78ef04d1f7 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:45:08.412320 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-75b7d358-5802-4ff4-ac2f-7d78ef04d1f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:45:08.433626 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2804b810-708c-4cc0-bb4c-a59788bb6eba None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:08.506440 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 300/615] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:45:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 10 12:45:08.509558 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59087761-b8e2-4953-b68f-4d75a582d39e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:08.510541 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59087761-b8e2-4953-b68f-4d75a582d39e None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:08.510957 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59087761-b8e2-4953-b68f-4d75a582d39e None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:08.511362 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59087761-b8e2-4953-b68f-4d75a582d39e None admin] PATH_INFO: `/v3/users/swift` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:08.524513 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59087761-b8e2-4953-b68f-4d75a582d39e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:08.526248 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59087761-b8e2-4953-b68f-4d75a582d39e None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:08.553739 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-59087761-b8e2-4953-b68f-4d75a582d39e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 10 12:45:08.553739 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:08.553739 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:08.553739 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:08.553739 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:08.553739 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:08.553739 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:08.553739 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:08.553739 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:08.553739 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:08.553739 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:08.553739 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:08.553739 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 10 12:45:08.553739 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 10 12:45:08.553739 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 10 12:45:08.553739 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 10 12:45:08.554993 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:08.554993 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:08.554993 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 10 12:45:08.554993 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 10 12:45:08.554993 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 10 12:45:08.554993 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 10 12:45:08.554993 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:08.554993 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:08.554993 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:08.554993 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:08.554993 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:08.554993 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:08.554993 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:08.554993 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:08.554993 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:08.554993 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:08.554993 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:08.554993 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:08.556210 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:08.556210 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:08.556210 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 10 12:45:08.556210 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 10 12:45:08.556210 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 10 12:45:08.556210 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 10 12:45:08.556210 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 10 12:45:08.556210 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 10 12:45:08.556210 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 10 12:45:08.556210 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:08.558587 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 316/616] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Jun 10 12:45:08 2022] GET /identity/v3/users/swift => generated 83 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:08.564384 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b7d358-5802-4ff4-ac2f-7d78ef04d1f7 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:08.572766 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59087761-b8e2-4953-b68f-4d75a582d39e None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:08.673009 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b7d358-5802-4ff4-ac2f-7d78ef04d1f7 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:08.678172 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:08.689296 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59087761-b8e2-4953-b68f-4d75a582d39e None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:08.699794 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:08.710399 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6856c5-d355-4aa7-a530-a4e388660af2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:08.711669 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6856c5-d355-4aa7-a530-a4e388660af2 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:08.712259 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6856c5-d355-4aa7-a530-a4e388660af2 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:08.712661 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6856c5-d355-4aa7-a530-a4e388660af2 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:08.713669 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6856c5-d355-4aa7-a530-a4e388660af2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:08.714133 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6856c5-d355-4aa7-a530-a4e388660af2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:08.715649 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6856c5-d355-4aa7-a530-a4e388660af2 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:08.729508 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343c2440-53b2-4e7c-b83e-e138a658f551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:08.730509 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343c2440-53b2-4e7c-b83e-e138a658f551 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:08.730941 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343c2440-53b2-4e7c-b83e-e138a658f551 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:08.731377 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343c2440-53b2-4e7c-b83e-e138a658f551 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:08.735114 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 301/617] 10.222.0.27 () {56 vars in 1135 bytes} [Fri Jun 10 12:45:08 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:08.743691 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6856c5-d355-4aa7-a530-a4e388660af2 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:08.749442 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343c2440-53b2-4e7c-b83e-e138a658f551 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:45:08.757539 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-343c2440-53b2-4e7c-b83e-e138a658f551 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 10 12:45:08.757539 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:08.757539 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:08.757539 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:08.757539 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:08.757539 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:08.757539 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:08.757539 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:08.757539 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:08.757539 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:08.757539 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:08.757539 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:08.757539 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:45:08.757539 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:45:08.757539 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:45:08.757539 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:45:08.759070 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:45:08.759070 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:45:08.759070 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:45:08.759070 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:45:08.759070 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:08.759070 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:08.759070 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:08.759070 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:08.759070 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:08.759070 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:08.759070 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:08.759070 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:08.759070 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:08.759070 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:08.759070 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:08.759070 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:08.759965 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:08.759965 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:08.759965 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:08.759965 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:08.759965 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:45:08.759965 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:45:08.759965 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:45:08.759965 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:45:08.759965 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:45:08.759965 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:45:08.759965 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 10 12:45:08.759965 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:08.762538 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 317/618] 10.222.0.27 () {56 vars in 1102 bytes} [Fri Jun 10 12:45:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:08.768405 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343c2440-53b2-4e7c-b83e-e138a658f551 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:08.879981 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343c2440-53b2-4e7c-b83e-e138a658f551 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:08.916679 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fadb0fc-bbbe-4cab-a063-a65846cda21e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:08.917733 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fadb0fc-bbbe-4cab-a063-a65846cda21e None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:08.918241 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fadb0fc-bbbe-4cab-a063-a65846cda21e None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:08.918731 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fadb0fc-bbbe-4cab-a063-a65846cda21e None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:08.919676 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fadb0fc-bbbe-4cab-a063-a65846cda21e None admin] RBAC: Adding query filter params (name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:08.920294 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fadb0fc-bbbe-4cab-a063-a65846cda21e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:08.922072 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fadb0fc-bbbe-4cab-a063-a65846cda21e None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:08.936324 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 318/619] 10.222.0.27 () {56 vars in 1103 bytes} [Fri Jun 10 12:45:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:08.937933 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fadb0fc-bbbe-4cab-a063-a65846cda21e None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:08.941020 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6856c5-d355-4aa7-a530-a4e388660af2 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:08.954752 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:08.977053 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd072fe7-7075-4146-8c16-4f25bf6c1328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:08.977905 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd072fe7-7075-4146-8c16-4f25bf6c1328 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:08.978182 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd072fe7-7075-4146-8c16-4f25bf6c1328 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:08.978512 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd072fe7-7075-4146-8c16-4f25bf6c1328 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:08.989091 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd072fe7-7075-4146-8c16-4f25bf6c1328 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:08.990557 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd072fe7-7075-4146-8c16-4f25bf6c1328 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:09.009130 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-bd072fe7-7075-4146-8c16-4f25bf6c1328 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:09.009130 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:09.009130 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:09.009130 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:09.009130 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:09.009130 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:09.009130 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:09.009130 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:09.009130 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:09.009130 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:09.009130 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:09.009130 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:09.009130 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 10 12:45:09.009130 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 10 12:45:09.009130 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 10 12:45:09.010496 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 10 12:45:09.010496 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:09.010496 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:09.010496 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:09.010496 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:09.010496 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:09.010496 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:09.010496 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:09.010496 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:09.010496 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:09.010496 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:09.010496 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:09.010496 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:09.010496 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:09.010496 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:09.010496 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:09.010496 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:09.011405 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 10 12:45:09.011405 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 10 12:45:09.011405 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:09.011405 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:09.011405 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 302/620] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:45:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 10 12:45:09.015964 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd072fe7-7075-4146-8c16-4f25bf6c1328 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:09.051764 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:09.057513 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fadb0fc-bbbe-4cab-a063-a65846cda21e None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:09.094860 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318e12e5-cc2a-40db-99e3-02baf40af7e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:09.096132 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318e12e5-cc2a-40db-99e3-02baf40af7e5 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:09.096838 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318e12e5-cc2a-40db-99e3-02baf40af7e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:09.097408 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318e12e5-cc2a-40db-99e3-02baf40af7e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:09.109812 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318e12e5-cc2a-40db-99e3-02baf40af7e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:09.111692 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318e12e5-cc2a-40db-99e3-02baf40af7e5 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:09.122097 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd072fe7-7075-4146-8c16-4f25bf6c1328 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:09.139463 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-318e12e5-cc2a-40db-99e3-02baf40af7e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:09.139463 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:09.139463 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:09.139463 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:09.139463 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:09.139463 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:09.139463 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:09.139463 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:09.139463 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:09.139463 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:09.139463 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:09.139463 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:09.139463 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 10 12:45:09.139463 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 10 12:45:09.139463 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 10 12:45:09.140436 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 10 12:45:09.140436 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:09.140436 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:09.140436 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:09.140436 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:09.140436 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:09.140436 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:09.140436 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:09.140436 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:09.140436 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:09.140436 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:09.140436 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:09.140436 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:09.140436 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:09.140436 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:09.140436 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:09.140436 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:09.141351 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 10 12:45:09.141351 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 10 12:45:09.141351 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:09.141351 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:09.143391 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 319/621] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:45:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 10 12:45:09.151317 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318e12e5-cc2a-40db-99e3-02baf40af7e5 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:09.169613 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524fe629-029a-4031-abd9-4d16b7dcbd3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:09.170300 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524fe629-029a-4031-abd9-4d16b7dcbd3a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:09.176954 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524fe629-029a-4031-abd9-4d16b7dcbd3a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:09.176954 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524fe629-029a-4031-abd9-4d16b7dcbd3a None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:09.176954 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524fe629-029a-4031-abd9-4d16b7dcbd3a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:09.176954 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524fe629-029a-4031-abd9-4d16b7dcbd3a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:09.176954 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524fe629-029a-4031-abd9-4d16b7dcbd3a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:09.205550 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 303/622] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Jun 10 12:45:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 12:45:09.217640 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524fe629-029a-4031-abd9-4d16b7dcbd3a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:09.270166 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:09.318292 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318e12e5-cc2a-40db-99e3-02baf40af7e5 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:09.356299 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d266cc3-7623-42fa-b04a-85f7280350ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:09.357254 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d266cc3-7623-42fa-b04a-85f7280350ae None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:09.357544 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d266cc3-7623-42fa-b04a-85f7280350ae None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:09.357822 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d266cc3-7623-42fa-b04a-85f7280350ae None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:09.358546 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d266cc3-7623-42fa-b04a-85f7280350ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:09.358917 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d266cc3-7623-42fa-b04a-85f7280350ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:09.360381 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d266cc3-7623-42fa-b04a-85f7280350ae None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:09.373699 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 320/623] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Jun 10 12:45:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 12:45:09.379437 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d266cc3-7623-42fa-b04a-85f7280350ae None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:09.491768 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d266cc3-7623-42fa-b04a-85f7280350ae None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:09.508227 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:09.518476 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:09.520158 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524fe629-029a-4031-abd9-4d16b7dcbd3a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:09.531387 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c4ad84-049c-4d31-a87c-f0e30ba2133b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:09.532229 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c4ad84-049c-4d31-a87c-f0e30ba2133b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:09.537321 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c4ad84-049c-4d31-a87c-f0e30ba2133b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:09.537321 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c4ad84-049c-4d31-a87c-f0e30ba2133b None admin] PATH_INFO: `/v3/users/glance` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:09.542236 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c4ad84-049c-4d31-a87c-f0e30ba2133b None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:09.543685 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c4ad84-049c-4d31-a87c-f0e30ba2133b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:09.560792 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-e4c4ad84-049c-4d31-a87c-f0e30ba2133b None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 10 12:45:09.560792 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:09.560792 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:09.560792 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:09.560792 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:09.560792 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:09.560792 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:09.560792 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:09.560792 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:09.560792 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:09.560792 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:09.560792 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:09.560792 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 10 12:45:09.560792 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 10 12:45:09.560792 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 10 12:45:09.560792 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 10 12:45:09.568266 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:09.568266 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:09.568266 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 10 12:45:09.568266 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 10 12:45:09.568266 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 10 12:45:09.568266 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 10 12:45:09.568266 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:09.568266 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:09.568266 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:09.568266 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:09.568266 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:09.568266 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:09.568266 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:09.568266 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:09.568266 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:09.568266 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:09.568266 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:09.568266 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:09.570531 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:09.570531 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:09.570531 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 10 12:45:09.570531 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 10 12:45:09.570531 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 10 12:45:09.570531 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 10 12:45:09.570531 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 10 12:45:09.570531 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 10 12:45:09.570531 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 10 12:45:09.570531 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:09.570531 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 321/624] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Jun 10 12:45:09 2022] GET /identity/v3/users/glance => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 10 12:45:09.570531 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c4ad84-049c-4d31-a87c-f0e30ba2133b None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:09.586455 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2c4fb4-6155-44e8-8db8-f1d30f519d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:09.587457 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2c4fb4-6155-44e8-8db8-f1d30f519d69 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:09.587840 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2c4fb4-6155-44e8-8db8-f1d30f519d69 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:09.588249 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2c4fb4-6155-44e8-8db8-f1d30f519d69 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:09.596201 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2c4fb4-6155-44e8-8db8-f1d30f519d69 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:09.597832 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2c4fb4-6155-44e8-8db8-f1d30f519d69 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:09.613727 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-4a2c4fb4-6155-44e8-8db8-f1d30f519d69 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 10 12:45:09.613727 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:09.613727 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:09.613727 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:09.613727 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:09.613727 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:09.613727 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:09.613727 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:09.613727 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:09.613727 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:09.613727 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:09.613727 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:09.613727 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 10 12:45:09.613727 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 10 12:45:09.613727 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 10 12:45:09.614901 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 10 12:45:09.614901 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:09.614901 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:09.614901 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:09.614901 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:09.614901 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:09.614901 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:09.614901 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:09.614901 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:09.614901 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:09.614901 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:09.614901 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:09.614901 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:09.614901 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:09.614901 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:09.614901 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:09.614901 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:09.616033 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 10 12:45:09.616033 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 10 12:45:09.616033 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 10 12:45:09.616033 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 10 12:45:09.616033 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 10 12:45:09.616033 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 10 12:45:09.616033 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 10 12:45:09.616033 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:09.620801 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 304/625] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Jun 10 12:45:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 401 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 10 12:45:09.628855 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2c4fb4-6155-44e8-8db8-f1d30f519d69 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:09.700396 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c4ad84-049c-4d31-a87c-f0e30ba2133b None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:09.739113 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975e5240-d053-4f0e-85c8-6d47566662ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:09.740272 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975e5240-d053-4f0e-85c8-6d47566662ca None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:09.740781 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975e5240-d053-4f0e-85c8-6d47566662ca None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:09.741315 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975e5240-d053-4f0e-85c8-6d47566662ca None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:09.742430 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975e5240-d053-4f0e-85c8-6d47566662ca None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:09.742936 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975e5240-d053-4f0e-85c8-6d47566662ca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:09.744701 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975e5240-d053-4f0e-85c8-6d47566662ca None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:09.765049 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 322/626] 10.222.0.27 () {56 vars in 1137 bytes} [Fri Jun 10 12:45:09 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 12:45:09.766822 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef7a3ec-72f8-46e8-b240-d4f897baac56 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:09.767435 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef7a3ec-72f8-46e8-b240-d4f897baac56 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:09.767937 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef7a3ec-72f8-46e8-b240-d4f897baac56 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:09.769793 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 323/627] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:45:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:45:09.770793 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef7a3ec-72f8-46e8-b240-d4f897baac56 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:09.779457 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2c4fb4-6155-44e8-8db8-f1d30f519d69 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:09.826408 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:09.828267 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca6feff-764e-4be2-827e-02caa3f92393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:09.828949 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca6feff-764e-4be2-827e-02caa3f92393 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:09.829235 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca6feff-764e-4be2-827e-02caa3f92393 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:09.829514 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca6feff-764e-4be2-827e-02caa3f92393 None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:09.830313 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca6feff-764e-4be2-827e-02caa3f92393 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:09.830625 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca6feff-764e-4be2-827e-02caa3f92393 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:09.832668 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca6feff-764e-4be2-827e-02caa3f92393 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:09.846033 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 305/628] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Jun 10 12:45:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:09.847372 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd25c988-3121-4063-a8b0-987366df1a4e None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:09.847675 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd25c988-3121-4063-a8b0-987366df1a4e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:09.847992 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd25c988-3121-4063-a8b0-987366df1a4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:09.864858 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-fd25c988-3121-4063-a8b0-987366df1a4e None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:45:09.866051 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-fd25c988-3121-4063-a8b0-987366df1a4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:45:09.878832 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:09.895121 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef7a3ec-72f8-46e8-b240-d4f897baac56 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:09.934157 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af76a552-a623-43a9-a380-71db27ac1a75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:09.935209 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af76a552-a623-43a9-a380-71db27ac1a75 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:09.935701 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af76a552-a623-43a9-a380-71db27ac1a75 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:09.936275 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af76a552-a623-43a9-a380-71db27ac1a75 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:09.948226 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 306/629] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:45:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 10 12:45:09.949141 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd25c988-3121-4063-a8b0-987366df1a4e None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:10.010171 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af76a552-a623-43a9-a380-71db27ac1a75 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:10.012143 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af76a552-a623-43a9-a380-71db27ac1a75 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:10.061504 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:10.079406 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd25c988-3121-4063-a8b0-987366df1a4e None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:10.115194 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144d8406-4b62-490d-8848-4198c26a177a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:10.116112 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144d8406-4b62-490d-8848-4198c26a177a None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:10.116398 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144d8406-4b62-490d-8848-4198c26a177a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:10.116678 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144d8406-4b62-490d-8848-4198c26a177a None admin] PATH_INFO: `/v3/projects/73ea5b23a2d34544ab493f4c5e4034b8/users/fb63b9cd74bf438aa25b72cef8337fec/roles/ea7e8f74523b4322857fb3f4307d171e` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:10.126742 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 324/630] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Jun 10 12:45:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 10 12:45:10.128421 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af76a552-a623-43a9-a380-71db27ac1a75 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:10.137332 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144d8406-4b62-490d-8848-4198c26a177a None admin] RBAC: Authorizing `identity:create_grant(project_id=73ea5b23a2d34544ab493f4c5e4034b8, user_id=fb63b9cd74bf438aa25b72cef8337fec, role_id=ea7e8f74523b4322857fb3f4307d171e)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:10.138787 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144d8406-4b62-490d-8848-4198c26a177a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:10.142025 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:10.229728 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 307/631] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Jun 10 12:45:09 2022] PUT /identity/v3/projects/73ea5b23a2d34544ab493f4c5e4034b8/users/fb63b9cd74bf438aa25b72cef8337fec/roles/ea7e8f74523b4322857fb3f4307d171e => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 12:45:10.232586 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af76a552-a623-43a9-a380-71db27ac1a75 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:10.241499 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542244c7-a68c-4869-ad29-de59995da886 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:10.241791 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542244c7-a68c-4869-ad29-de59995da886 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:10.242755 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542244c7-a68c-4869-ad29-de59995da886 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:10.244419 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 308/632] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:45:10 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:45:10.244868 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542244c7-a68c-4869-ad29-de59995da886 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:10.287483 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07f2731-04f6-4cf4-a5d6-6b7f8def774c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:10.288607 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07f2731-04f6-4cf4-a5d6-6b7f8def774c None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:10.289035 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07f2731-04f6-4cf4-a5d6-6b7f8def774c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:10.289321 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07f2731-04f6-4cf4-a5d6-6b7f8def774c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:10.305911 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07f2731-04f6-4cf4-a5d6-6b7f8def774c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:45:10.316809 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-b07f2731-04f6-4cf4-a5d6-6b7f8def774c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 10 12:45:10.316809 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:10.316809 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:10.316809 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:10.316809 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:10.316809 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:10.316809 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:10.316809 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:10.316809 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:10.316809 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:10.316809 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:10.316809 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:10.316809 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:45:10.316809 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:45:10.316809 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:45:10.316809 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:45:10.317826 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:45:10.317826 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:45:10.317826 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:45:10.317826 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:45:10.317826 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:10.317826 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:10.317826 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:10.317826 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:10.317826 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:10.317826 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:10.317826 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:10.317826 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:10.317826 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:10.317826 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:10.317826 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:10.317826 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:10.318673 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:10.318673 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:10.318673 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:10.318673 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:10.318673 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:45:10.318673 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:45:10.318673 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:45:10.318673 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:45:10.318673 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:45:10.318673 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:45:10.318673 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 10 12:45:10.318673 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:10.319815 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 325/633] 10.222.0.27 () {56 vars in 1102 bytes} [Fri Jun 10 12:45:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:10.322164 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480f63a5-e4c8-46e2-a1ae-5a107abc62eb None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:10.322462 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480f63a5-e4c8-46e2-a1ae-5a107abc62eb None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:10.323031 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480f63a5-e4c8-46e2-a1ae-5a107abc62eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:10.329985 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:10.347473 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-480f63a5-e4c8-46e2-a1ae-5a107abc62eb None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:45:10.349388 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-480f63a5-e4c8-46e2-a1ae-5a107abc62eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:45:10.376178 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542244c7-a68c-4869-ad29-de59995da886 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:10.419408 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e3bb9f-b7b7-4a96-8def-d9ecd598ce52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:10.420369 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e3bb9f-b7b7-4a96-8def-d9ecd598ce52 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:10.420662 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e3bb9f-b7b7-4a96-8def-d9ecd598ce52 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:10.420944 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e3bb9f-b7b7-4a96-8def-d9ecd598ce52 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:10.435166 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e3bb9f-b7b7-4a96-8def-d9ecd598ce52 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:10.437510 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e3bb9f-b7b7-4a96-8def-d9ecd598ce52 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:10.449541 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 326/634] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:45:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 10 12:45:10.451149 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480f63a5-e4c8-46e2-a1ae-5a107abc62eb None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:10.458452 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-b5e3bb9f-b7b7-4a96-8def-d9ecd598ce52 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:10.458452 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:10.458452 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:10.458452 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:10.458452 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:10.458452 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:10.458452 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:10.458452 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:10.458452 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:10.458452 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:10.458452 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:10.458452 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:10.458452 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 10 12:45:10.458452 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 10 12:45:10.458452 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 10 12:45:10.459382 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 10 12:45:10.459382 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:10.459382 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:10.459382 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:10.459382 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:10.459382 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:10.459382 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:10.459382 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:10.459382 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:10.459382 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:10.459382 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:10.459382 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:10.459382 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:10.459382 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:10.459382 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:10.459382 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:10.459382 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:10.460372 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 10 12:45:10.460372 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 10 12:45:10.460372 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:10.460372 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:10.464178 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 309/635] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:45:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:10.465012 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e3bb9f-b7b7-4a96-8def-d9ecd598ce52 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:10.516911 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:10.571508 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e3bb9f-b7b7-4a96-8def-d9ecd598ce52 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:10.590028 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480f63a5-e4c8-46e2-a1ae-5a107abc62eb None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:10.612568 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07240815-6fdd-4b72-9ef7-7140bb910bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:10.613256 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07240815-6fdd-4b72-9ef7-7140bb910bb3 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:10.613545 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07240815-6fdd-4b72-9ef7-7140bb910bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:10.613833 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07240815-6fdd-4b72-9ef7-7140bb910bb3 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:10.614630 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07240815-6fdd-4b72-9ef7-7140bb910bb3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:10.615476 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07240815-6fdd-4b72-9ef7-7140bb910bb3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:10.617800 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07240815-6fdd-4b72-9ef7-7140bb910bb3 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:10.626806 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:10.634382 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8095e92b-9cbb-436a-b05f-0335cc374f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:10.641127 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8095e92b-9cbb-436a-b05f-0335cc374f72 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:10.641127 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8095e92b-9cbb-436a-b05f-0335cc374f72 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:10.641127 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8095e92b-9cbb-436a-b05f-0335cc374f72 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:10.641127 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8095e92b-9cbb-436a-b05f-0335cc374f72 None admin] RBAC: Adding query filter params (name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:10.641127 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8095e92b-9cbb-436a-b05f-0335cc374f72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:10.641127 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8095e92b-9cbb-436a-b05f-0335cc374f72 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:10.641541 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 310/636] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Jun 10 12:45:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:10.641541 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07240815-6fdd-4b72-9ef7-7140bb910bb3 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:10.654660 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 327/637] 10.222.0.27 () {56 vars in 1103 bytes} [Fri Jun 10 12:45:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:10.654660 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8095e92b-9cbb-436a-b05f-0335cc374f72 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:10.767287 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07240815-6fdd-4b72-9ef7-7140bb910bb3 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:10.772730 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8095e92b-9cbb-436a-b05f-0335cc374f72 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:10.777064 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:10.807494 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ddfd96-4b37-437c-8bf2-14a0d4161e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:10.808321 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ddfd96-4b37-437c-8bf2-14a0d4161e0c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:10.808606 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ddfd96-4b37-437c-8bf2-14a0d4161e0c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:10.808899 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ddfd96-4b37-437c-8bf2-14a0d4161e0c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:10.814623 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cc11ca-e60a-47b6-a92b-70cbe9856d4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:10.815802 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cc11ca-e60a-47b6-a92b-70cbe9856d4d None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:10.816340 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cc11ca-e60a-47b6-a92b-70cbe9856d4d None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:10.816845 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cc11ca-e60a-47b6-a92b-70cbe9856d4d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:10.821478 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ddfd96-4b37-437c-8bf2-14a0d4161e0c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:45:10.825445 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cc11ca-e60a-47b6-a92b-70cbe9856d4d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:10.827237 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cc11ca-e60a-47b6-a92b-70cbe9856d4d None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:10.829384 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-18ddfd96-4b37-437c-8bf2-14a0d4161e0c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 10 12:45:10.829384 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:10.829384 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:10.829384 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:10.829384 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:10.829384 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:10.829384 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:10.829384 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:10.829384 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:10.829384 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:10.829384 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:10.829384 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:10.829384 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:45:10.829384 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:45:10.829384 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:45:10.829384 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:45:10.830363 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:45:10.830363 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:45:10.830363 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:45:10.830363 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:45:10.830363 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:10.830363 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:10.830363 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:10.830363 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:10.830363 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:10.830363 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:10.830363 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:10.830363 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:10.830363 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:10.830363 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:10.830363 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:10.830363 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:10.831223 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:10.831223 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:10.831223 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:10.831223 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:10.831223 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:45:10.831223 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:45:10.831223 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:45:10.831223 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:45:10.831223 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:45:10.831223 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:45:10.831223 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 10 12:45:10.831223 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:10.835276 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 311/638] 10.222.0.27 () {56 vars in 1102 bytes} [Fri Jun 10 12:45:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:10.836723 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ddfd96-4b37-437c-8bf2-14a0d4161e0c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:10.847512 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-42cc11ca-e60a-47b6-a92b-70cbe9856d4d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 10 12:45:10.847512 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:10.847512 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:10.847512 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:10.847512 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:10.847512 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:10.847512 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:10.847512 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:10.847512 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:10.847512 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:10.847512 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:10.847512 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:10.847512 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 10 12:45:10.847512 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 10 12:45:10.847512 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 10 12:45:10.848462 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 10 12:45:10.848462 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:10.848462 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:10.848462 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:10.848462 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:10.848462 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:10.848462 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:10.848462 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:10.848462 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:10.848462 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:10.848462 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:10.848462 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:10.848462 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:10.848462 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:10.848462 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:10.848462 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:10.848462 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:10.849371 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 10 12:45:10.849371 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 10 12:45:10.849371 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 10 12:45:10.849371 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 10 12:45:10.849371 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 10 12:45:10.849371 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 10 12:45:10.849371 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 10 12:45:10.849371 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:10.856842 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 328/639] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Jun 10 12:45:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:10.857969 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cc11ca-e60a-47b6-a92b-70cbe9856d4d None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:10.917204 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:10.945685 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ddfd96-4b37-437c-8bf2-14a0d4161e0c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:10.965398 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cc11ca-e60a-47b6-a92b-70cbe9856d4d None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:10.981758 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768b4df1-6f2d-4d60-8bbd-fb83a4b846a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:10.982642 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768b4df1-6f2d-4d60-8bbd-fb83a4b846a5 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:10.983017 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768b4df1-6f2d-4d60-8bbd-fb83a4b846a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:10.983305 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768b4df1-6f2d-4d60-8bbd-fb83a4b846a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:10.994700 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768b4df1-6f2d-4d60-8bbd-fb83a4b846a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:10.996186 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768b4df1-6f2d-4d60-8bbd-fb83a4b846a5 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:11.001451 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753fa109-4930-45b5-966b-a1100dc7ec99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:11.002129 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753fa109-4930-45b5-966b-a1100dc7ec99 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:11.002414 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753fa109-4930-45b5-966b-a1100dc7ec99 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:11.002692 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753fa109-4930-45b5-966b-a1100dc7ec99 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:11.003584 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753fa109-4930-45b5-966b-a1100dc7ec99 None admin] RBAC: Adding query filter params (name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:11.003743 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753fa109-4930-45b5-966b-a1100dc7ec99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:11.005136 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753fa109-4930-45b5-966b-a1100dc7ec99 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:11.013729 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 329/640] 10.222.0.27 () {56 vars in 1103 bytes} [Fri Jun 10 12:45:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:11.014609 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-768b4df1-6f2d-4d60-8bbd-fb83a4b846a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:11.014609 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:11.014609 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:11.014609 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:11.014609 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:11.014609 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:11.014609 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:11.014609 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:11.014609 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:11.014609 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:11.014609 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:11.014609 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:11.014609 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 10 12:45:11.014609 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 10 12:45:11.014609 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 10 12:45:11.015603 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 10 12:45:11.015603 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:11.015603 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:11.015603 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:11.015603 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:11.015603 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:11.015603 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:11.015603 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:11.015603 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:11.015603 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:11.015603 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:11.015603 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:11.015603 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:11.015603 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:11.015603 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:11.015603 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:11.015603 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:11.017068 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 10 12:45:11.017068 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 10 12:45:11.017068 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:11.017068 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:11.018689 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753fa109-4930-45b5-966b-a1100dc7ec99 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:11.020260 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 312/641] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:45:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:11.021409 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8396cd6d-cc89-44ce-8b9b-debcc711781f None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:11.021700 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8396cd6d-cc89-44ce-8b9b-debcc711781f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:11.021980 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8396cd6d-cc89-44ce-8b9b-debcc711781f None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:11.028052 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 313/642] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:45:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:45:11.028052 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8396cd6d-cc89-44ce-8b9b-debcc711781f None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:11.051082 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:11.126049 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753fa109-4930-45b5-966b-a1100dc7ec99 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:11.133265 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8396cd6d-cc89-44ce-8b9b-debcc711781f None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:11.143085 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:11.161236 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f318cd6-dad8-4301-8f55-06a420719c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:11.162151 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f318cd6-dad8-4301-8f55-06a420719c41 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:11.162433 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f318cd6-dad8-4301-8f55-06a420719c41 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:11.162709 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f318cd6-dad8-4301-8f55-06a420719c41 None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:11.163489 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f318cd6-dad8-4301-8f55-06a420719c41 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:11.163798 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f318cd6-dad8-4301-8f55-06a420719c41 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:11.165941 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f318cd6-dad8-4301-8f55-06a420719c41 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:11.175617 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10197afd-a316-4ea5-adf0-01ffa19e94dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:11.176443 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10197afd-a316-4ea5-adf0-01ffa19e94dc None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:11.176727 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10197afd-a316-4ea5-adf0-01ffa19e94dc None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:11.177000 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10197afd-a316-4ea5-adf0-01ffa19e94dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:11.178711 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 330/643] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Jun 10 12:45:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:11.179373 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f318cd6-dad8-4301-8f55-06a420719c41 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:11.193329 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10197afd-a316-4ea5-adf0-01ffa19e94dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:11.194815 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10197afd-a316-4ea5-adf0-01ffa19e94dc None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:11.212844 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-10197afd-a316-4ea5-adf0-01ffa19e94dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:11.212844 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:11.212844 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:11.212844 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:11.212844 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:11.212844 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:11.212844 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:11.212844 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:11.212844 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:11.212844 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:11.212844 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:11.212844 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:11.212844 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 10 12:45:11.212844 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 10 12:45:11.212844 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 10 12:45:11.213936 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 10 12:45:11.213936 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:11.213936 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:11.213936 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:11.213936 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:11.213936 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:11.213936 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:11.213936 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:11.213936 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:11.213936 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:11.213936 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:11.213936 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:11.213936 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:11.213936 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:11.213936 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:11.213936 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:11.213936 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:11.215242 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 10 12:45:11.215242 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 10 12:45:11.215242 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:11.215242 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:11.218733 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 314/644] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:45:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:11.220004 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d483aef1-55b3-4e75-b63e-20a42795e9f5 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:11.220298 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d483aef1-55b3-4e75-b63e-20a42795e9f5 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:11.220580 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d483aef1-55b3-4e75-b63e-20a42795e9f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:11.236954 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-d483aef1-55b3-4e75-b63e-20a42795e9f5 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:45:11.238128 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d483aef1-55b3-4e75-b63e-20a42795e9f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:45:11.287558 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f318cd6-dad8-4301-8f55-06a420719c41 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:11.307912 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 315/645] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:45:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 10 12:45:11.307912 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d483aef1-55b3-4e75-b63e-20a42795e9f5 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:11.307912 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:11.327373 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae872aed-6ba7-49f1-ae62-9e7b4f5bb559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:11.328229 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae872aed-6ba7-49f1-ae62-9e7b4f5bb559 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:11.328624 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae872aed-6ba7-49f1-ae62-9e7b4f5bb559 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:11.329021 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae872aed-6ba7-49f1-ae62-9e7b4f5bb559 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:11.329874 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae872aed-6ba7-49f1-ae62-9e7b4f5bb559 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:11.330292 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae872aed-6ba7-49f1-ae62-9e7b4f5bb559 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:11.331792 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae872aed-6ba7-49f1-ae62-9e7b4f5bb559 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:11.345589 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 331/646] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Jun 10 12:45:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:11.345589 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae872aed-6ba7-49f1-ae62-9e7b4f5bb559 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:11.388322 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:11.422854 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d483aef1-55b3-4e75-b63e-20a42795e9f5 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:11.457335 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae872aed-6ba7-49f1-ae62-9e7b4f5bb559 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:11.459137 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae567405-aec1-4945-9c38-0be665e349af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:11.460031 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae567405-aec1-4945-9c38-0be665e349af None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:11.460382 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae567405-aec1-4945-9c38-0be665e349af None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:11.460670 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae567405-aec1-4945-9c38-0be665e349af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:11.462118 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae567405-aec1-4945-9c38-0be665e349af None admin] RBAC: Adding query filter params (role.id=ea7e8f74523b4322857fb3f4307d171e, scope.project.id=73ea5b23a2d34544ab493f4c5e4034b8, user.id=6d3eb01a98274050b930616b98998c06) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:11.462451 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae567405-aec1-4945-9c38-0be665e349af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:11.463884 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae567405-aec1-4945-9c38-0be665e349af None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:11.479890 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 316/647] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Jun 10 12:45:11 2022] GET /identity/v3/role_assignments?user.id=6d3eb01a98274050b930616b98998c06&scope.project.id=73ea5b23a2d34544ab493f4c5e4034b8&role.id=ea7e8f74523b4322857fb3f4307d171e => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:11.479890 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae567405-aec1-4945-9c38-0be665e349af None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:11.511195 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2161c2eb-bdd0-437d-9147-ad98a7fe2508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:11.512011 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2161c2eb-bdd0-437d-9147-ad98a7fe2508 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:11.512404 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2161c2eb-bdd0-437d-9147-ad98a7fe2508 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:11.512783 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2161c2eb-bdd0-437d-9147-ad98a7fe2508 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:11.513631 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2161c2eb-bdd0-437d-9147-ad98a7fe2508 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:11.514043 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2161c2eb-bdd0-437d-9147-ad98a7fe2508 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:11.515573 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2161c2eb-bdd0-437d-9147-ad98a7fe2508 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:11.533739 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 332/648] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Jun 10 12:45:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:11.534867 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2161c2eb-bdd0-437d-9147-ad98a7fe2508 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:11.568861 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:11.616570 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae567405-aec1-4945-9c38-0be665e349af None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:11.647134 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:11.654430 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2161c2eb-bdd0-437d-9147-ad98a7fe2508 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:11.668602 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e698bfb-d5f6-43f1-a320-f734fd1427db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:11.669414 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e698bfb-d5f6-43f1-a320-f734fd1427db None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:11.669700 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e698bfb-d5f6-43f1-a320-f734fd1427db None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:11.669979 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e698bfb-d5f6-43f1-a320-f734fd1427db None admin] PATH_INFO: `/v3/roles/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:11.681587 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e698bfb-d5f6-43f1-a320-f734fd1427db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:45:11.689605 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-9e698bfb-d5f6-43f1-a320-f734fd1427db None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 10 12:45:11.689605 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:11.689605 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:11.689605 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:11.689605 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:11.689605 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:11.689605 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:11.689605 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:11.689605 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:11.689605 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:11.689605 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:11.689605 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:11.689605 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:45:11.689605 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:45:11.689605 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:45:11.689605 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:45:11.690759 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:45:11.690759 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:45:11.690759 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:45:11.690759 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:45:11.690759 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:11.690759 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:11.690759 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:11.690759 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:11.690759 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:11.690759 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:11.690759 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:11.690759 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:11.690759 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:11.690759 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:11.690759 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:11.690759 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:11.691929 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:11.691929 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:11.691929 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:11.691929 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:11.691929 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:45:11.691929 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:45:11.691929 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:45:11.691929 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:45:11.691929 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:45:11.691929 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:45:11.691929 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 10 12:45:11.691929 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:11.694371 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 317/649] 10.222.0.27 () {56 vars in 1102 bytes} [Fri Jun 10 12:45:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:11.694371 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e698bfb-d5f6-43f1-a320-f734fd1427db None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:11.705513 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcbdf57-03d3-404b-8122-8dc405a1ee44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:11.706408 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcbdf57-03d3-404b-8122-8dc405a1ee44 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:11.706766 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcbdf57-03d3-404b-8122-8dc405a1ee44 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:11.707115 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcbdf57-03d3-404b-8122-8dc405a1ee44 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:11.716375 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcbdf57-03d3-404b-8122-8dc405a1ee44 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:11.717922 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcbdf57-03d3-404b-8122-8dc405a1ee44 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:11.732751 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-bfcbdf57-03d3-404b-8122-8dc405a1ee44 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 10 12:45:11.732751 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:11.732751 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:11.732751 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:11.732751 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:11.732751 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:11.732751 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:11.732751 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:11.732751 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:11.732751 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:11.732751 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:11.732751 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:11.732751 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 10 12:45:11.732751 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 10 12:45:11.732751 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 10 12:45:11.732751 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 10 12:45:11.733908 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:11.733908 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:11.733908 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 10 12:45:11.733908 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 10 12:45:11.733908 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 10 12:45:11.733908 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 10 12:45:11.733908 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:11.733908 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:11.733908 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:11.733908 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:11.733908 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:11.733908 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:11.733908 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:11.733908 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:11.733908 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:11.733908 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:11.733908 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:11.733908 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:11.735233 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:11.735233 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:11.735233 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 10 12:45:11.735233 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 10 12:45:11.735233 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 10 12:45:11.735233 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 10 12:45:11.735233 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 10 12:45:11.735233 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 10 12:45:11.735233 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 10 12:45:11.735233 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:11.735233 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 333/650] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Jun 10 12:45:11 2022] GET /identity/v3/users/cinder => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:11.735233 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcbdf57-03d3-404b-8122-8dc405a1ee44 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:11.827120 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e698bfb-d5f6-43f1-a320-f734fd1427db None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:11.854838 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcbdf57-03d3-404b-8122-8dc405a1ee44 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:11.859459 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:11.871117 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e32fb58-c994-43f7-b6ef-4f6d5b62cd9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:11.872154 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e32fb58-c994-43f7-b6ef-4f6d5b62cd9b None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:11.872617 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e32fb58-c994-43f7-b6ef-4f6d5b62cd9b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:11.873283 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e32fb58-c994-43f7-b6ef-4f6d5b62cd9b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:11.883746 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e32fb58-c994-43f7-b6ef-4f6d5b62cd9b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:11.885486 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e32fb58-c994-43f7-b6ef-4f6d5b62cd9b None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:11.902475 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-0e32fb58-c994-43f7-b6ef-4f6d5b62cd9b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 10 12:45:11.902475 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:11.902475 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:11.902475 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:11.902475 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:11.902475 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:11.902475 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:11.902475 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:11.902475 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:11.902475 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:11.902475 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:11.902475 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:11.902475 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 10 12:45:11.902475 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 10 12:45:11.902475 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 10 12:45:11.902475 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 10 12:45:11.903647 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:11.903647 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:11.903647 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 10 12:45:11.903647 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 10 12:45:11.903647 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 10 12:45:11.903647 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 10 12:45:11.903647 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:11.903647 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:11.903647 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:11.903647 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:11.903647 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:11.903647 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:11.903647 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:11.903647 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:11.903647 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:11.903647 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:11.903647 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:11.903647 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:11.904763 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:11.904763 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:11.904763 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 10 12:45:11.904763 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 10 12:45:11.904763 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 10 12:45:11.904763 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 10 12:45:11.904763 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 10 12:45:11.904763 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 10 12:45:11.904763 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 10 12:45:11.904763 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:11.906035 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868f755f-c33c-419f-96d2-797efe901b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:11.906708 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f755f-c33c-419f-96d2-797efe901b85 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:11.907010 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f755f-c33c-419f-96d2-797efe901b85 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:11.907379 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868f755f-c33c-419f-96d2-797efe901b85 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:11.907577 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 318/651] 10.222.0.27 () {56 vars in 1093 bytes} [Fri Jun 10 12:45:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:11.908042 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868f755f-c33c-419f-96d2-797efe901b85 None admin] RBAC: Adding query filter params (name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:11.908348 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868f755f-c33c-419f-96d2-797efe901b85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:11.908991 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e32fb58-c994-43f7-b6ef-4f6d5b62cd9b None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:11.909721 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868f755f-c33c-419f-96d2-797efe901b85 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:11.923979 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 334/652] 10.222.0.27 () {56 vars in 1103 bytes} [Fri Jun 10 12:45:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:11.923979 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868f755f-c33c-419f-96d2-797efe901b85 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:11.944561 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:12.026189 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e32fb58-c994-43f7-b6ef-4f6d5b62cd9b None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:12.046898 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868f755f-c33c-419f-96d2-797efe901b85 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:12.066385 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6410f64-c75b-40ba-bc59-d5529ad3d46c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:12.067589 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6410f64-c75b-40ba-bc59-d5529ad3d46c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:12.068498 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6410f64-c75b-40ba-bc59-d5529ad3d46c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:12.068938 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6410f64-c75b-40ba-bc59-d5529ad3d46c None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:12.069940 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6410f64-c75b-40ba-bc59-d5529ad3d46c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:12.070482 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6410f64-c75b-40ba-bc59-d5529ad3d46c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:12.072990 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6410f64-c75b-40ba-bc59-d5529ad3d46c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:12.087847 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c839c1-f52d-4b42-a1b0-763f8eacce53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:12.088604 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c839c1-f52d-4b42-a1b0-763f8eacce53 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:12.088906 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c839c1-f52d-4b42-a1b0-763f8eacce53 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:12.089178 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c839c1-f52d-4b42-a1b0-763f8eacce53 None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:12.090021 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c839c1-f52d-4b42-a1b0-763f8eacce53 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:12.090332 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c839c1-f52d-4b42-a1b0-763f8eacce53 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:12.091766 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c839c1-f52d-4b42-a1b0-763f8eacce53 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:12.102883 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 319/653] 10.222.0.27 () {56 vars in 1137 bytes} [Fri Jun 10 12:45:11 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:12.104089 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6410f64-c75b-40ba-bc59-d5529ad3d46c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:12.118848 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 335/654] 10.222.0.27 () {56 vars in 1133 bytes} [Fri Jun 10 12:45:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:12.119776 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c839c1-f52d-4b42-a1b0-763f8eacce53 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:12.172372 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:12.208751 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:12.233511 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c839c1-f52d-4b42-a1b0-763f8eacce53 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:12.243491 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6410f64-c75b-40ba-bc59-d5529ad3d46c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:12.276437 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b83ba6-06ee-4d06-95f4-bb3ea89aa20f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:12.277393 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b83ba6-06ee-4d06-95f4-bb3ea89aa20f None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:12.277858 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b83ba6-06ee-4d06-95f4-bb3ea89aa20f None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:12.278312 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b83ba6-06ee-4d06-95f4-bb3ea89aa20f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:12.289337 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4754de87-3ca1-4e92-8bb7-c776785ed89b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:12.290161 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4754de87-3ca1-4e92-8bb7-c776785ed89b None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:12.290446 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4754de87-3ca1-4e92-8bb7-c776785ed89b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:12.290723 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4754de87-3ca1-4e92-8bb7-c776785ed89b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:12.305730 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4754de87-3ca1-4e92-8bb7-c776785ed89b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:12.307850 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4754de87-3ca1-4e92-8bb7-c776785ed89b None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:12.332059 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-4754de87-3ca1-4e92-8bb7-c776785ed89b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:12.332059 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:12.332059 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:12.332059 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:12.332059 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:12.332059 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:12.332059 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:12.332059 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:12.332059 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:12.332059 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:12.332059 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:12.332059 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:12.332059 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 10 12:45:12.332059 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 10 12:45:12.332059 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 10 12:45:12.333018 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 10 12:45:12.333018 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:12.333018 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:12.333018 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:12.333018 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:12.333018 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:12.333018 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:12.333018 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:12.333018 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:12.333018 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:12.333018 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:12.333018 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:12.333018 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:12.333018 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:12.333018 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:12.333018 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:12.333018 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:12.333941 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 10 12:45:12.333941 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 10 12:45:12.333941 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:12.333941 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:12.333941 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 320/655] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:45:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:12.334676 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4754de87-3ca1-4e92-8bb7-c776785ed89b None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:12.358421 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b83ba6-06ee-4d06-95f4-bb3ea89aa20f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:12.360340 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b83ba6-06ee-4d06-95f4-bb3ea89aa20f None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:12.446596 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:12.457455 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:12.463540 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4754de87-3ca1-4e92-8bb7-c776785ed89b None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:12.474360 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 336/656] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Jun 10 12:45:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 10 12:45:12.474360 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b83ba6-06ee-4d06-95f4-bb3ea89aa20f None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:12.503200 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8300247-c82d-4e7b-a794-7fa98b84fe8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:12.503850 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8300247-c82d-4e7b-a794-7fa98b84fe8e None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:12.504160 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8300247-c82d-4e7b-a794-7fa98b84fe8e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:12.504450 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8300247-c82d-4e7b-a794-7fa98b84fe8e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:12.570376 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b83ba6-06ee-4d06-95f4-bb3ea89aa20f None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:12.573121 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8300247-c82d-4e7b-a794-7fa98b84fe8e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:12.574646 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8300247-c82d-4e7b-a794-7fa98b84fe8e None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:12.607037 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb1045d-e39c-4505-8f93-fd91f9900fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:12.607691 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb1045d-e39c-4505-8f93-fd91f9900fc2 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:12.608005 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb1045d-e39c-4505-8f93-fd91f9900fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:12.608282 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb1045d-e39c-4505-8f93-fd91f9900fc2 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:12.609000 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb1045d-e39c-4505-8f93-fd91f9900fc2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:12.609310 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb1045d-e39c-4505-8f93-fd91f9900fc2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:12.610675 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb1045d-e39c-4505-8f93-fd91f9900fc2 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:12.621825 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 337/657] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Jun 10 12:45:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:12.623047 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce97aebf-4b5c-41d4-88d5-4992fdcd27c5 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:12.623334 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce97aebf-4b5c-41d4-88d5-4992fdcd27c5 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:12.623612 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce97aebf-4b5c-41d4-88d5-4992fdcd27c5 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:12.625005 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 338/658] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:45:12 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:45:12.625524 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce97aebf-4b5c-41d4-88d5-4992fdcd27c5 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:12.665393 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:12.688847 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:12.700072 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 321/659] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Jun 10 12:45:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 10 12:45:12.700072 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8300247-c82d-4e7b-a794-7fa98b84fe8e None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:12.740373 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce97aebf-4b5c-41d4-88d5-4992fdcd27c5 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:12.777818 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ff941b-e32e-45b2-aa05-9b5fd412a621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:12.778614 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ff941b-e32e-45b2-aa05-9b5fd412a621 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:12.778895 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ff941b-e32e-45b2-aa05-9b5fd412a621 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:12.779175 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ff941b-e32e-45b2-aa05-9b5fd412a621 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:12.790099 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ff941b-e32e-45b2-aa05-9b5fd412a621 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:12.791542 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ff941b-e32e-45b2-aa05-9b5fd412a621 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:12.805335 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8300247-c82d-4e7b-a794-7fa98b84fe8e None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:12.808760 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-02ff941b-e32e-45b2-aa05-9b5fd412a621 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:12.808760 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:12.808760 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:12.808760 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:12.808760 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:12.808760 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:12.808760 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:12.808760 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:12.808760 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:12.808760 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:12.808760 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:12.808760 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:12.808760 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 10 12:45:12.808760 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 10 12:45:12.808760 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 10 12:45:12.809651 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 10 12:45:12.809651 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:12.809651 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:12.809651 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:12.809651 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:12.809651 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:12.809651 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:12.809651 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:12.809651 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:12.809651 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:12.809651 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:12.809651 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:12.809651 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:12.809651 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:12.809651 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:12.809651 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:12.809651 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:12.810640 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 10 12:45:12.810640 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 10 12:45:12.810640 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:12.810640 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:12.810640 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 339/660] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:45:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:12.815497 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c2548f-1a0b-4902-81d3-ab37362fad5b None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:12.815794 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c2548f-1a0b-4902-81d3-ab37362fad5b None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:12.816098 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c2548f-1a0b-4902-81d3-ab37362fad5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:12.836875 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-02c2548f-1a0b-4902-81d3-ab37362fad5b None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:45:12.839088 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-02c2548f-1a0b-4902-81d3-ab37362fad5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:45:12.842237 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e12de7-3881-4a1d-b7bb-df2b82dfe4ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:12.843029 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e12de7-3881-4a1d-b7bb-df2b82dfe4ad None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:12.843314 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e12de7-3881-4a1d-b7bb-df2b82dfe4ad None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:12.843590 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e12de7-3881-4a1d-b7bb-df2b82dfe4ad None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:12.851158 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e12de7-3881-4a1d-b7bb-df2b82dfe4ad None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:12.852661 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e12de7-3881-4a1d-b7bb-df2b82dfe4ad None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:12.868852 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-b2e12de7-3881-4a1d-b7bb-df2b82dfe4ad None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 10 12:45:12.868852 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:12.868852 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:12.868852 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:12.868852 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:12.868852 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:12.868852 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:12.868852 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:12.868852 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:12.868852 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:12.868852 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:12.868852 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:12.868852 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 10 12:45:12.868852 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 10 12:45:12.868852 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 10 12:45:12.868852 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 10 12:45:12.869808 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:12.869808 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:12.869808 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 10 12:45:12.869808 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 10 12:45:12.869808 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 10 12:45:12.869808 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 10 12:45:12.869808 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:12.869808 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:12.869808 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:12.869808 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:12.869808 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:12.869808 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:12.869808 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:12.869808 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:12.869808 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:12.869808 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:12.869808 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:12.869808 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:12.870775 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:12.870775 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:12.870775 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 10 12:45:12.870775 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 10 12:45:12.870775 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 10 12:45:12.870775 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 10 12:45:12.870775 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 10 12:45:12.870775 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 10 12:45:12.870775 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 10 12:45:12.870775 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:12.873158 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 322/661] 10.222.0.27 () {56 vars in 1102 bytes} [Fri Jun 10 12:45:12 2022] GET /identity/v3/users/neutron => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:12.874172 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e12de7-3881-4a1d-b7bb-df2b82dfe4ad None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:12.921574 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 340/662] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:45:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 10 12:45:12.922695 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c2548f-1a0b-4902-81d3-ab37362fad5b None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:12.941788 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:12.952320 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:12.988696 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e12de7-3881-4a1d-b7bb-df2b82dfe4ad None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:13.025389 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70596913-50d7-4f34-99aa-977533e9aa37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:13.026173 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70596913-50d7-4f34-99aa-977533e9aa37 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:13.026462 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70596913-50d7-4f34-99aa-977533e9aa37 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:13.026735 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70596913-50d7-4f34-99aa-977533e9aa37 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:13.036307 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c2548f-1a0b-4902-81d3-ab37362fad5b None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:13.037716 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70596913-50d7-4f34-99aa-977533e9aa37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:13.039135 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70596913-50d7-4f34-99aa-977533e9aa37 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:13.062524 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-70596913-50d7-4f34-99aa-977533e9aa37 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:13.062524 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:13.062524 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:13.062524 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:13.062524 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:13.062524 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:13.062524 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:13.062524 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:13.062524 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:13.062524 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:13.062524 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:13.062524 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:13.062524 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 10 12:45:13.062524 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 10 12:45:13.062524 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 10 12:45:13.063580 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 10 12:45:13.063580 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:13.063580 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:13.063580 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:13.063580 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:13.063580 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:13.063580 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:13.063580 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:13.063580 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:13.063580 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:13.063580 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:13.063580 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:13.063580 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:13.063580 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:13.063580 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:13.063580 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:13.063580 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:13.064783 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 10 12:45:13.064783 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 10 12:45:13.064783 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:13.064783 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:13.068262 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 323/663] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:45:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:13.069087 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70596913-50d7-4f34-99aa-977533e9aa37 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:13.075656 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b37dc1d-fe99-4346-bdc5-6b859acc4b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:13.076361 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b37dc1d-fe99-4346-bdc5-6b859acc4b8b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:13.076644 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b37dc1d-fe99-4346-bdc5-6b859acc4b8b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:13.076914 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b37dc1d-fe99-4346-bdc5-6b859acc4b8b None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:13.077632 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b37dc1d-fe99-4346-bdc5-6b859acc4b8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:13.077936 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b37dc1d-fe99-4346-bdc5-6b859acc4b8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:13.079360 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b37dc1d-fe99-4346-bdc5-6b859acc4b8b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:13.091722 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 341/664] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Jun 10 12:45:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:13.096647 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b37dc1d-fe99-4346-bdc5-6b859acc4b8b None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:13.173127 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70596913-50d7-4f34-99aa-977533e9aa37 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:13.182992 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:13.183491 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:13.208104 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b37dc1d-fe99-4346-bdc5-6b859acc4b8b None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:13.208450 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32dd335-6d43-4e0a-88b3-63a391998c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:13.209178 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32dd335-6d43-4e0a-88b3-63a391998c45 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:13.209495 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32dd335-6d43-4e0a-88b3-63a391998c45 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:13.209729 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32dd335-6d43-4e0a-88b3-63a391998c45 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:13.210859 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32dd335-6d43-4e0a-88b3-63a391998c45 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:13.210859 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32dd335-6d43-4e0a-88b3-63a391998c45 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:13.212363 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32dd335-6d43-4e0a-88b3-63a391998c45 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:13.235114 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 324/665] 10.222.0.27 () {56 vars in 1139 bytes} [Fri Jun 10 12:45:13 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:13.235114 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32dd335-6d43-4e0a-88b3-63a391998c45 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:13.248393 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622a3154-0713-4f90-aba2-0e8df7fe82fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:13.249270 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622a3154-0713-4f90-aba2-0e8df7fe82fa None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:13.249558 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622a3154-0713-4f90-aba2-0e8df7fe82fa None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:13.249835 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622a3154-0713-4f90-aba2-0e8df7fe82fa None admin] PATH_INFO: `/v3/roles/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:13.260842 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622a3154-0713-4f90-aba2-0e8df7fe82fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:45:13.268443 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-622a3154-0713-4f90-aba2-0e8df7fe82fa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 10 12:45:13.268443 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:13.268443 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:13.268443 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:13.268443 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:13.268443 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:13.268443 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:13.268443 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:13.268443 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:13.268443 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:13.268443 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:13.268443 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:13.268443 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:45:13.268443 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:45:13.268443 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:45:13.268443 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:45:13.269450 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:45:13.269450 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:45:13.269450 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:45:13.269450 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:45:13.269450 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:13.269450 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:13.269450 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:13.269450 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:13.269450 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:13.269450 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:13.269450 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:13.269450 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:13.269450 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:13.269450 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:13.269450 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:13.269450 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:13.270326 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:13.270326 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:13.270326 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:13.270326 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:13.270326 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:45:13.270326 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:45:13.270326 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:45:13.270326 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:45:13.270326 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:45:13.270326 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:45:13.270326 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 10 12:45:13.270326 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:13.270326 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 342/666] 10.222.0.27 () {56 vars in 1102 bytes} [Fri Jun 10 12:45:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:13.271618 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622a3154-0713-4f90-aba2-0e8df7fe82fa None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:13.343874 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32dd335-6d43-4e0a-88b3-63a391998c45 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:13.378049 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622a3154-0713-4f90-aba2-0e8df7fe82fa None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:13.379892 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a200593-d783-4526-b993-cd2b7cbb9f07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:13.380576 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a200593-d783-4526-b993-cd2b7cbb9f07 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:13.380859 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a200593-d783-4526-b993-cd2b7cbb9f07 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:13.381164 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a200593-d783-4526-b993-cd2b7cbb9f07 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:13.381879 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a200593-d783-4526-b993-cd2b7cbb9f07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:13.382181 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a200593-d783-4526-b993-cd2b7cbb9f07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:13.384164 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a200593-d783-4526-b993-cd2b7cbb9f07 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:13.395775 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 325/667] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Jun 10 12:45:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:13.396605 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a200593-d783-4526-b993-cd2b7cbb9f07 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:13.420690 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149c70ec-8b95-43b1-a90e-736985b5b7e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:13.421720 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149c70ec-8b95-43b1-a90e-736985b5b7e6 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:13.422077 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149c70ec-8b95-43b1-a90e-736985b5b7e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:13.422425 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149c70ec-8b95-43b1-a90e-736985b5b7e6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:13.425905 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:13.429305 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149c70ec-8b95-43b1-a90e-736985b5b7e6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:13.431315 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149c70ec-8b95-43b1-a90e-736985b5b7e6 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:13.433277 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:13.447482 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-149c70ec-8b95-43b1-a90e-736985b5b7e6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 10 12:45:13.447482 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:13.447482 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:13.447482 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:13.447482 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:13.447482 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:13.447482 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:13.447482 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:13.447482 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:13.447482 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:13.447482 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:13.447482 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:13.447482 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 10 12:45:13.447482 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 10 12:45:13.447482 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 10 12:45:13.448406 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 10 12:45:13.448406 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:13.448406 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:13.448406 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:13.448406 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:13.448406 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:13.448406 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:13.448406 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:13.448406 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:13.448406 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:13.448406 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:13.448406 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:13.448406 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:13.448406 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:13.448406 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:13.448406 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:13.448406 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:13.449366 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 10 12:45:13.449366 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 10 12:45:13.449366 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 10 12:45:13.449366 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 10 12:45:13.449366 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 10 12:45:13.449366 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 10 12:45:13.449366 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 10 12:45:13.449366 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:13.449366 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 343/668] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Jun 10 12:45:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:13.449855 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149c70ec-8b95-43b1-a90e-736985b5b7e6 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:13.508236 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a200593-d783-4526-b993-cd2b7cbb9f07 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:13.542378 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607bc282-0ff5-425b-ad20-e83cae83731c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:13.543017 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607bc282-0ff5-425b-ad20-e83cae83731c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:13.543297 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607bc282-0ff5-425b-ad20-e83cae83731c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:13.543583 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607bc282-0ff5-425b-ad20-e83cae83731c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:13.550176 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149c70ec-8b95-43b1-a90e-736985b5b7e6 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:13.584145 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc89b872-cce8-4e25-adbc-265314450617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:13.584817 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc89b872-cce8-4e25-adbc-265314450617 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:13.585103 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc89b872-cce8-4e25-adbc-265314450617 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:13.585377 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc89b872-cce8-4e25-adbc-265314450617 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:13.586153 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc89b872-cce8-4e25-adbc-265314450617 None admin] RBAC: Adding query filter params (name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:13.586456 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc89b872-cce8-4e25-adbc-265314450617 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:13.587815 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc89b872-cce8-4e25-adbc-265314450617 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:13.596158 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 344/669] 10.222.0.27 () {56 vars in 1103 bytes} [Fri Jun 10 12:45:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:13.597050 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc89b872-cce8-4e25-adbc-265314450617 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:13.623854 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607bc282-0ff5-425b-ad20-e83cae83731c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:13.625346 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607bc282-0ff5-425b-ad20-e83cae83731c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:13.652401 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:13.658248 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:13.705255 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc89b872-cce8-4e25-adbc-265314450617 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:13.738405 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 326/670] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Jun 10 12:45:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 10 12:45:13.739265 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607bc282-0ff5-425b-ad20-e83cae83731c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:13.741834 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb372f2-e4d6-4db6-8340-bbdad8048007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:13.743303 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb372f2-e4d6-4db6-8340-bbdad8048007 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:13.743587 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb372f2-e4d6-4db6-8340-bbdad8048007 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:13.743887 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb372f2-e4d6-4db6-8340-bbdad8048007 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:13.752236 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb372f2-e4d6-4db6-8340-bbdad8048007 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:13.753848 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb372f2-e4d6-4db6-8340-bbdad8048007 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:13.768353 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-abb372f2-e4d6-4db6-8340-bbdad8048007 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 10 12:45:13.768353 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:13.768353 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:13.768353 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:13.768353 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:13.768353 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:13.768353 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:13.768353 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:13.768353 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:13.768353 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:13.768353 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:13.768353 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:13.768353 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 10 12:45:13.768353 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 10 12:45:13.768353 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 10 12:45:13.769252 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 10 12:45:13.769252 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:13.769252 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:13.769252 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:13.769252 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:13.769252 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:13.769252 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:13.769252 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:13.769252 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:13.769252 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:13.769252 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:13.769252 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:13.769252 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:13.769252 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:13.769252 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:13.769252 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:13.769252 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:13.770628 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 10 12:45:13.770628 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 10 12:45:13.770628 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 10 12:45:13.770628 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 10 12:45:13.770628 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 10 12:45:13.770628 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 10 12:45:13.770628 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 10 12:45:13.770628 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:13.770628 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 345/671] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Jun 10 12:45:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:13.770628 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb372f2-e4d6-4db6-8340-bbdad8048007 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:13.841728 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607bc282-0ff5-425b-ad20-e83cae83731c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:13.867844 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:13.869429 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb372f2-e4d6-4db6-8340-bbdad8048007 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:13.875836 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00dce62-9b0b-46a5-ad79-43d3edbf850c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:13.876525 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00dce62-9b0b-46a5-ad79-43d3edbf850c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:13.876811 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00dce62-9b0b-46a5-ad79-43d3edbf850c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:13.877090 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00dce62-9b0b-46a5-ad79-43d3edbf850c None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:13.877874 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00dce62-9b0b-46a5-ad79-43d3edbf850c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:13.878184 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00dce62-9b0b-46a5-ad79-43d3edbf850c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:13.879623 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00dce62-9b0b-46a5-ad79-43d3edbf850c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:13.889934 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:13.890829 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 327/672] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Jun 10 12:45:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:13.891606 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00dce62-9b0b-46a5-ad79-43d3edbf850c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:13.905333 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e4a498-7c6f-4d5f-a8f8-75788935312a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:13.906377 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e4a498-7c6f-4d5f-a8f8-75788935312a None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:13.906377 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e4a498-7c6f-4d5f-a8f8-75788935312a None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:13.907018 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e4a498-7c6f-4d5f-a8f8-75788935312a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:13.920951 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e4a498-7c6f-4d5f-a8f8-75788935312a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:13.922385 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e4a498-7c6f-4d5f-a8f8-75788935312a None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:13.939496 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-f1e4a498-7c6f-4d5f-a8f8-75788935312a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:13.939496 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:13.939496 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:13.939496 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:13.939496 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:13.939496 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:13.939496 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:13.939496 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:13.939496 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:13.939496 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:13.939496 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:13.939496 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:13.939496 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 10 12:45:13.939496 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 10 12:45:13.939496 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 10 12:45:13.940442 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 10 12:45:13.940442 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:13.940442 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:13.940442 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:13.940442 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:13.940442 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:13.940442 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:13.940442 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:13.940442 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:13.940442 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:13.940442 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:13.940442 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:13.940442 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:13.940442 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:13.940442 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:13.940442 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:13.940442 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:13.941357 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 10 12:45:13.941357 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 10 12:45:13.941357 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:13.941357 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:13.941357 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 346/673] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:45:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:13.941902 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1dd657-5542-4e1a-862e-3086c6b9127f None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:13.942155 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1dd657-5542-4e1a-862e-3086c6b9127f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:13.942405 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1dd657-5542-4e1a-862e-3086c6b9127f None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:13.943632 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 347/674] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:45:13 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:45:13.944333 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1dd657-5542-4e1a-862e-3086c6b9127f None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:14.000105 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00dce62-9b0b-46a5-ad79-43d3edbf850c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:14.032373 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd189f9-8b8e-46ee-8191-7ecca50ff759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:14.033102 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd189f9-8b8e-46ee-8191-7ecca50ff759 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:14.033477 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd189f9-8b8e-46ee-8191-7ecca50ff759 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:14.033672 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd189f9-8b8e-46ee-8191-7ecca50ff759 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:14.042447 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1dd657-5542-4e1a-862e-3086c6b9127f None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:14.044772 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd189f9-8b8e-46ee-8191-7ecca50ff759 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:14.046487 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd189f9-8b8e-46ee-8191-7ecca50ff759 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:14.063520 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-8bd189f9-8b8e-46ee-8191-7ecca50ff759 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:14.063520 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:14.063520 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:14.063520 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:14.063520 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:14.063520 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:14.063520 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:14.063520 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:14.063520 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:14.063520 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:14.063520 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:14.063520 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:14.063520 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 10 12:45:14.063520 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 10 12:45:14.063520 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 10 12:45:14.064582 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 10 12:45:14.064582 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:14.064582 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:14.064582 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:14.064582 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:14.064582 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:14.064582 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:14.064582 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:14.064582 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:14.064582 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:14.064582 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:14.064582 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:14.064582 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:14.064582 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:14.064582 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:14.064582 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:14.064582 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:14.065485 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 10 12:45:14.065485 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 10 12:45:14.065485 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:14.065485 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:14.065485 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 328/675] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:45:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:14.065899 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd189f9-8b8e-46ee-8191-7ecca50ff759 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:14.076590 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6760b547-6716-45c2-bb55-d01009b6b747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:14.077252 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6760b547-6716-45c2-bb55-d01009b6b747 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:14.077542 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6760b547-6716-45c2-bb55-d01009b6b747 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:14.077818 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6760b547-6716-45c2-bb55-d01009b6b747 None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:14.078631 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6760b547-6716-45c2-bb55-d01009b6b747 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:14.078940 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6760b547-6716-45c2-bb55-d01009b6b747 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:14.080413 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6760b547-6716-45c2-bb55-d01009b6b747 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:14.092149 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 348/676] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Jun 10 12:45:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:14.092914 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6760b547-6716-45c2-bb55-d01009b6b747 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:14.106137 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:14.124737 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:14.166939 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd189f9-8b8e-46ee-8191-7ecca50ff759 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:14.190609 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6760b547-6716-45c2-bb55-d01009b6b747 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:14.199238 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7795e1-e081-405f-8dcc-c58b9dad0706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:14.199908 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7795e1-e081-405f-8dcc-c58b9dad0706 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:14.200192 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7795e1-e081-405f-8dcc-c58b9dad0706 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:14.200471 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7795e1-e081-405f-8dcc-c58b9dad0706 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:14.201341 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7795e1-e081-405f-8dcc-c58b9dad0706 None admin] RBAC: Adding query filter params (role.id=ea7e8f74523b4322857fb3f4307d171e, scope.project.id=73ea5b23a2d34544ab493f4c5e4034b8, user.id=c6cc95a563b941ac92ab283aa0d5a5d5) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:14.201671 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7795e1-e081-405f-8dcc-c58b9dad0706 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:14.203092 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7795e1-e081-405f-8dcc-c58b9dad0706 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:14.208753 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 329/677] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Jun 10 12:45:14 2022] GET /identity/v3/role_assignments?user.id=c6cc95a563b941ac92ab283aa0d5a5d5&scope.project.id=73ea5b23a2d34544ab493f4c5e4034b8&role.id=ea7e8f74523b4322857fb3f4307d171e => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:14.209891 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44e63c4-5563-4cf6-bfb2-0cf5d4c98bca None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:14.210175 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44e63c4-5563-4cf6-bfb2-0cf5d4c98bca None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:14.210452 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44e63c4-5563-4cf6-bfb2-0cf5d4c98bca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:14.228465 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-d44e63c4-5563-4cf6-bfb2-0cf5d4c98bca None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:45:14.229695 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d44e63c4-5563-4cf6-bfb2-0cf5d4c98bca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:45:14.239557 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d27f45-55e0-4ab2-91d9-69f9d7a7a3cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:14.240288 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d27f45-55e0-4ab2-91d9-69f9d7a7a3cb None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:14.240572 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d27f45-55e0-4ab2-91d9-69f9d7a7a3cb None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:14.240845 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d27f45-55e0-4ab2-91d9-69f9d7a7a3cb None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:14.242054 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d27f45-55e0-4ab2-91d9-69f9d7a7a3cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:14.242364 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d27f45-55e0-4ab2-91d9-69f9d7a7a3cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:14.244160 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d27f45-55e0-4ab2-91d9-69f9d7a7a3cb None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:14.261942 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 349/678] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Jun 10 12:45:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:14.261942 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d27f45-55e0-4ab2-91d9-69f9d7a7a3cb None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:14.328129 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 330/679] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:45:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 10 12:45:14.329327 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44e63c4-5563-4cf6-bfb2-0cf5d4c98bca None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:14.362025 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:14.369955 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:14.371318 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d27f45-55e0-4ab2-91d9-69f9d7a7a3cb None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:14.409421 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f89a202-ecad-4f87-b878-7734c06d6ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:14.410086 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f89a202-ecad-4f87-b878-7734c06d6ba2 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:14.410372 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f89a202-ecad-4f87-b878-7734c06d6ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:14.410647 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f89a202-ecad-4f87-b878-7734c06d6ba2 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:14.411374 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f89a202-ecad-4f87-b878-7734c06d6ba2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:14.412028 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f89a202-ecad-4f87-b878-7734c06d6ba2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:14.413401 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f89a202-ecad-4f87-b878-7734c06d6ba2 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:14.425552 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 350/680] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Jun 10 12:45:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:14.426363 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f89a202-ecad-4f87-b878-7734c06d6ba2 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:14.449067 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44e63c4-5563-4cf6-bfb2-0cf5d4c98bca None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:14.485361 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703824de-7b05-4a92-b0b0-1479b4430690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:14.486015 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703824de-7b05-4a92-b0b0-1479b4430690 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:14.486298 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703824de-7b05-4a92-b0b0-1479b4430690 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:14.486571 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703824de-7b05-4a92-b0b0-1479b4430690 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:14.487393 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703824de-7b05-4a92-b0b0-1479b4430690 None admin] RBAC: Adding query filter params (role.id=ea7e8f74523b4322857fb3f4307d171e, scope.project.id=73ea5b23a2d34544ab493f4c5e4034b8, user.id=8bd6caa461e84100a3d5fe5af5d80f3e) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:14.487702 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703824de-7b05-4a92-b0b0-1479b4430690 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:14.489166 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703824de-7b05-4a92-b0b0-1479b4430690 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:14.495125 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 331/681] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Jun 10 12:45:14 2022] GET /identity/v3/role_assignments?user.id=8bd6caa461e84100a3d5fe5af5d80f3e&scope.project.id=73ea5b23a2d34544ab493f4c5e4034b8&role.id=ea7e8f74523b4322857fb3f4307d171e => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:14.495916 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703824de-7b05-4a92-b0b0-1479b4430690 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:14.558977 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f89a202-ecad-4f87-b878-7734c06d6ba2 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:14.608515 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee65863b-148e-4427-9734-4a488513f751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:14.609631 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee65863b-148e-4427-9734-4a488513f751 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:14.609919 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee65863b-148e-4427-9734-4a488513f751 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:14.610196 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee65863b-148e-4427-9734-4a488513f751 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:14.616517 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:14.623504 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee65863b-148e-4427-9734-4a488513f751 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:14.624026 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703824de-7b05-4a92-b0b0-1479b4430690 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:14.625029 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee65863b-148e-4427-9734-4a488513f751 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:14.651045 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-ee65863b-148e-4427-9734-4a488513f751 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 10 12:45:14.651045 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:14.651045 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:14.651045 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:14.651045 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:14.651045 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:14.651045 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:14.651045 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:14.651045 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:14.651045 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:14.651045 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:14.651045 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:14.651045 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 10 12:45:14.651045 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 10 12:45:14.651045 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 10 12:45:14.651045 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 10 12:45:14.652530 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:14.652530 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:14.652530 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 10 12:45:14.652530 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 10 12:45:14.652530 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 10 12:45:14.652530 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 10 12:45:14.652530 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:14.652530 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:14.652530 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:14.652530 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:14.652530 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:14.652530 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:14.652530 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:14.652530 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:14.652530 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:14.652530 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:14.652530 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:14.652530 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:14.653527 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:14.653527 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:14.653527 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 10 12:45:14.653527 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 10 12:45:14.653527 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 10 12:45:14.653527 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 10 12:45:14.653527 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 10 12:45:14.653527 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 10 12:45:14.653527 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 10 12:45:14.653527 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:14.658200 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 351/682] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Jun 10 12:45:14 2022] GET /identity/v3/users/swift => generated 83 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:14.659148 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee65863b-148e-4427-9734-4a488513f751 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:14.678014 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbb37ab-2920-4944-87b6-4ed97c1d7a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:14.678643 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbb37ab-2920-4944-87b6-4ed97c1d7a2e None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:14.678928 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbb37ab-2920-4944-87b6-4ed97c1d7a2e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:14.679208 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbb37ab-2920-4944-87b6-4ed97c1d7a2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:14.681870 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:14.704521 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbb37ab-2920-4944-87b6-4ed97c1d7a2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:14.707884 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbb37ab-2920-4944-87b6-4ed97c1d7a2e None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:14.728034 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-7cbb37ab-2920-4944-87b6-4ed97c1d7a2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:14.728034 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:14.728034 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:14.728034 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:14.728034 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:14.728034 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:14.728034 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:14.728034 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:14.728034 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:14.728034 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:14.728034 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:14.728034 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:14.728034 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 10 12:45:14.728034 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 10 12:45:14.728034 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 10 12:45:14.729095 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 10 12:45:14.729095 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:14.729095 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:14.729095 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:14.729095 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:14.729095 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:14.729095 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:14.729095 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:14.729095 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:14.729095 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:14.729095 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:14.729095 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:14.729095 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:14.729095 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:14.729095 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:14.729095 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:14.729095 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:14.730602 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 10 12:45:14.730602 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 10 12:45:14.730602 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:14.730602 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:14.730602 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 332/683] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:45:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:14.730602 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbb37ab-2920-4944-87b6-4ed97c1d7a2e None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:14.794189 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee65863b-148e-4427-9734-4a488513f751 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:14.843443 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22dd42e-123b-4829-9c69-e71771e41946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:14.845380 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22dd42e-123b-4829-9c69-e71771e41946 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:14.846071 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22dd42e-123b-4829-9c69-e71771e41946 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:14.846860 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22dd42e-123b-4829-9c69-e71771e41946 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:14.853210 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbb37ab-2920-4944-87b6-4ed97c1d7a2e None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:14.858330 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22dd42e-123b-4829-9c69-e71771e41946 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:14.860150 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22dd42e-123b-4829-9c69-e71771e41946 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:14.876632 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-e22dd42e-123b-4829-9c69-e71771e41946 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 10 12:45:14.876632 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:14.876632 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:14.876632 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:14.876632 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:14.876632 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:14.876632 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:14.876632 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:14.876632 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:14.876632 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:14.876632 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:14.876632 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:14.876632 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 10 12:45:14.876632 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 10 12:45:14.876632 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 10 12:45:14.877767 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 10 12:45:14.877767 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:14.877767 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:14.877767 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:14.877767 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:14.877767 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:14.877767 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:14.877767 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:14.877767 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:14.877767 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:14.877767 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:14.877767 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:14.877767 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:14.877767 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:14.877767 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:14.877767 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:14.877767 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:14.879073 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 10 12:45:14.879073 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 10 12:45:14.879073 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 10 12:45:14.879073 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 10 12:45:14.879073 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 10 12:45:14.879073 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 10 12:45:14.879073 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 10 12:45:14.879073 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:14.879073 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 352/684] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Jun 10 12:45:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:14.883787 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22dd42e-123b-4829-9c69-e71771e41946 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:14.900131 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b33035-cfe5-4459-b898-9fd1b14467c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:14.900772 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b33035-cfe5-4459-b898-9fd1b14467c6 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:14.901058 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b33035-cfe5-4459-b898-9fd1b14467c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:14.901337 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b33035-cfe5-4459-b898-9fd1b14467c6 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:14.902238 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b33035-cfe5-4459-b898-9fd1b14467c6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:14.902552 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b33035-cfe5-4459-b898-9fd1b14467c6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:14.904111 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b33035-cfe5-4459-b898-9fd1b14467c6 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:14.923401 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 333/685] 10.222.0.27 () {56 vars in 1135 bytes} [Fri Jun 10 12:45:14 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:14.924377 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b33035-cfe5-4459-b898-9fd1b14467c6 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:14.934362 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:14.983241 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:15.022745 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22dd42e-123b-4829-9c69-e71771e41946 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:15.054308 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b33035-cfe5-4459-b898-9fd1b14467c6 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:15.059985 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c60839-fed4-4378-9302-d6585cfbb194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:15.059985 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c60839-fed4-4378-9302-d6585cfbb194 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:15.060346 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c60839-fed4-4378-9302-d6585cfbb194 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:15.060731 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c60839-fed4-4378-9302-d6585cfbb194 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:15.061555 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c60839-fed4-4378-9302-d6585cfbb194 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:15.061984 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c60839-fed4-4378-9302-d6585cfbb194 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:15.063505 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c60839-fed4-4378-9302-d6585cfbb194 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:15.076478 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 353/686] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Jun 10 12:45:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:15.077184 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c60839-fed4-4378-9302-d6585cfbb194 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:15.110927 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561bdf90-f7e4-4687-97d8-9090ad41e516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:15.111519 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561bdf90-f7e4-4687-97d8-9090ad41e516 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:15.111924 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561bdf90-f7e4-4687-97d8-9090ad41e516 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:15.112304 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561bdf90-f7e4-4687-97d8-9090ad41e516 None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:15.113223 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561bdf90-f7e4-4687-97d8-9090ad41e516 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:15.113634 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561bdf90-f7e4-4687-97d8-9090ad41e516 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:15.115157 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561bdf90-f7e4-4687-97d8-9090ad41e516 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:15.128614 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 334/687] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Jun 10 12:45:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:15.129899 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561bdf90-f7e4-4687-97d8-9090ad41e516 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:15.204275 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:15.211219 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c60839-fed4-4378-9302-d6585cfbb194 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:15.234680 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561bdf90-f7e4-4687-97d8-9090ad41e516 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:15.253199 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:15.260664 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528f7c19-a4fe-47e2-b687-8f71b073a47b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:15.262014 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f7c19-a4fe-47e2-b687-8f71b073a47b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:15.262787 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f7c19-a4fe-47e2-b687-8f71b073a47b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:15.263287 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f7c19-a4fe-47e2-b687-8f71b073a47b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:15.269274 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15d9792-b82b-45ec-ba7b-42725ecb7e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:15.270008 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15d9792-b82b-45ec-ba7b-42725ecb7e16 None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:15.270295 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15d9792-b82b-45ec-ba7b-42725ecb7e16 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:15.270578 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15d9792-b82b-45ec-ba7b-42725ecb7e16 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:15.271277 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15d9792-b82b-45ec-ba7b-42725ecb7e16 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:15.272744 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15d9792-b82b-45ec-ba7b-42725ecb7e16 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:15.341539 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528f7c19-a4fe-47e2-b687-8f71b073a47b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:15.343037 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528f7c19-a4fe-47e2-b687-8f71b073a47b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:15.361263 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 335/688] 10.222.0.27 () {60 vars in 1132 bytes} [Fri Jun 10 12:45:15 2022] POST /identity/v3/users => generated 254 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 12:45:15.362023 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15d9792-b82b-45ec-ba7b-42725ecb7e16 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:15.518983 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15d9792-b82b-45ec-ba7b-42725ecb7e16 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:15.550529 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:15.564924 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 354/689] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Jun 10 12:45:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 10 12:45:15.570901 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528f7c19-a4fe-47e2-b687-8f71b073a47b None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:15.581527 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4a05f1-6e19-404f-a99d-75c4635d1c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:15.581527 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4a05f1-6e19-404f-a99d-75c4635d1c5e None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:15.581527 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4a05f1-6e19-404f-a99d-75c4635d1c5e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:15.581527 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4a05f1-6e19-404f-a99d-75c4635d1c5e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:15.582132 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4a05f1-6e19-404f-a99d-75c4635d1c5e None admin] RBAC: Adding query filter params (role.id=ea7e8f74523b4322857fb3f4307d171e, scope.project.id=73ea5b23a2d34544ab493f4c5e4034b8, user.id=bc4415461a974c24ad7105a97c7d915b) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:15.582132 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4a05f1-6e19-404f-a99d-75c4635d1c5e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:15.582132 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4a05f1-6e19-404f-a99d-75c4635d1c5e None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:15.582132 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 336/690] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Jun 10 12:45:15 2022] GET /identity/v3/role_assignments?user.id=bc4415461a974c24ad7105a97c7d915b&scope.project.id=73ea5b23a2d34544ab493f4c5e4034b8&role.id=ea7e8f74523b4322857fb3f4307d171e => generated 608 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:15.615786 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:15.694380 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528f7c19-a4fe-47e2-b687-8f71b073a47b None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:15.736660 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1ebecc-d1a3-4c2b-b205-34118c098ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:15.737559 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1ebecc-d1a3-4c2b-b205-34118c098ba6 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:15.738055 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1ebecc-d1a3-4c2b-b205-34118c098ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:15.738524 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1ebecc-d1a3-4c2b-b205-34118c098ba6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:15.756083 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1ebecc-d1a3-4c2b-b205-34118c098ba6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:15.758946 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1ebecc-d1a3-4c2b-b205-34118c098ba6 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:15.785444 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-5b1ebecc-d1a3-4c2b-b205-34118c098ba6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:15.785444 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:15.785444 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:15.785444 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:15.785444 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:15.785444 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:15.785444 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:15.785444 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:15.785444 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:15.785444 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:15.785444 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:15.785444 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:15.785444 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 10 12:45:15.785444 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 10 12:45:15.785444 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 10 12:45:15.786420 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 10 12:45:15.786420 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:15.786420 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:15.786420 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:15.786420 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:15.786420 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:15.786420 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:15.786420 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:15.786420 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:15.786420 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:15.786420 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:15.786420 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:15.786420 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:15.786420 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:15.786420 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:15.786420 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:15.786420 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:15.787608 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 10 12:45:15.787608 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 10 12:45:15.787608 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:15.787608 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:15.787608 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 355/691] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:45:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 10 12:45:15.793732 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4a05f1-6e19-404f-a99d-75c4635d1c5e None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:15.904963 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4a05f1-6e19-404f-a99d-75c4635d1c5e None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:15.940663 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc92b776-d94e-4873-98ce-124686eb8f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:15.941397 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc92b776-d94e-4873-98ce-124686eb8f0a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:15.941685 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc92b776-d94e-4873-98ce-124686eb8f0a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:15.941967 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc92b776-d94e-4873-98ce-124686eb8f0a None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:15.942751 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc92b776-d94e-4873-98ce-124686eb8f0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:15.943066 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc92b776-d94e-4873-98ce-124686eb8f0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:15.944465 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc92b776-d94e-4873-98ce-124686eb8f0a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:15.962523 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 337/692] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Jun 10 12:45:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:15.963884 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1ebecc-d1a3-4c2b-b205-34118c098ba6 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:16.064642 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:16.076456 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1ebecc-d1a3-4c2b-b205-34118c098ba6 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:16.113485 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9205fc7-04f7-4801-90c8-7f60c6b2eaee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:16.114285 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9205fc7-04f7-4801-90c8-7f60c6b2eaee None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:16.114571 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9205fc7-04f7-4801-90c8-7f60c6b2eaee None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:16.114849 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9205fc7-04f7-4801-90c8-7f60c6b2eaee None admin] PATH_INFO: `/v3/projects/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:16.122476 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9205fc7-04f7-4801-90c8-7f60c6b2eaee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:16.123876 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9205fc7-04f7-4801-90c8-7f60c6b2eaee None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:16.139324 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-d9205fc7-04f7-4801-90c8-7f60c6b2eaee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 10 12:45:16.139324 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:16.139324 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:16.139324 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:16.139324 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:16.139324 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:16.139324 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:16.139324 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:16.139324 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:16.139324 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:16.139324 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:16.139324 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:16.139324 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 10 12:45:16.139324 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 10 12:45:16.139324 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 10 12:45:16.140379 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 10 12:45:16.140379 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:16.140379 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:16.140379 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:16.140379 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:16.140379 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:16.140379 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:16.140379 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:16.140379 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:16.140379 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:16.140379 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:16.140379 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:16.140379 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:16.140379 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:16.140379 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:16.140379 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:16.140379 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:16.141281 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 10 12:45:16.141281 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 10 12:45:16.141281 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 10 12:45:16.141281 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 10 12:45:16.141281 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 10 12:45:16.141281 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 10 12:45:16.141281 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 10 12:45:16.141281 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:16.143201 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 356/693] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Jun 10 12:45:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:16.146664 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc92b776-d94e-4873-98ce-124686eb8f0a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:16.198492 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:16.254491 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc92b776-d94e-4873-98ce-124686eb8f0a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:16.290723 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57deae3d-54ce-4e22-bf55-10abea1f1837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:16.291394 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57deae3d-54ce-4e22-bf55-10abea1f1837 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:16.291691 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57deae3d-54ce-4e22-bf55-10abea1f1837 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:16.291996 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57deae3d-54ce-4e22-bf55-10abea1f1837 None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:16.292789 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57deae3d-54ce-4e22-bf55-10abea1f1837 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:16.293105 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57deae3d-54ce-4e22-bf55-10abea1f1837 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:16.294536 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57deae3d-54ce-4e22-bf55-10abea1f1837 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:16.311170 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 338/694] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Jun 10 12:45:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:16.313689 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9205fc7-04f7-4801-90c8-7f60c6b2eaee None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:16.421701 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9205fc7-04f7-4801-90c8-7f60c6b2eaee None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:16.457450 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5f97ec-4b7f-4599-97d5-70422b13459e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:16.458117 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5f97ec-4b7f-4599-97d5-70422b13459e None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:16.458404 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5f97ec-4b7f-4599-97d5-70422b13459e None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:16.458686 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5f97ec-4b7f-4599-97d5-70422b13459e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:16.459532 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5f97ec-4b7f-4599-97d5-70422b13459e None admin] RBAC: Adding query filter params (role.id=ea7e8f74523b4322857fb3f4307d171e, scope.project.id=73ea5b23a2d34544ab493f4c5e4034b8, user.id=fb63b9cd74bf438aa25b72cef8337fec) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:16.459851 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5f97ec-4b7f-4599-97d5-70422b13459e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:16.461424 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5f97ec-4b7f-4599-97d5-70422b13459e None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:16.472261 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 357/695] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Jun 10 12:45:16 2022] GET /identity/v3/role_assignments?user.id=fb63b9cd74bf438aa25b72cef8337fec&scope.project.id=73ea5b23a2d34544ab493f4c5e4034b8&role.id=ea7e8f74523b4322857fb3f4307d171e => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 12:45:16.652171 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc60e089-2cb8-4620-b904-65007d0fcfba None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:16.652521 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc60e089-2cb8-4620-b904-65007d0fcfba None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:16.652799 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc60e089-2cb8-4620-b904-65007d0fcfba None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:16.654536 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 339/696] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:45:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:45:16.661958 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bdad24-8df9-4f04-906a-acb4a6c2f858 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:16.662250 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bdad24-8df9-4f04-906a-acb4a6c2f858 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:16.662533 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bdad24-8df9-4f04-906a-acb4a6c2f858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:16.680358 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-d1bdad24-8df9-4f04-906a-acb4a6c2f858 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:45:16.681496 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d1bdad24-8df9-4f04-906a-acb4a6c2f858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:45:16.683597 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:16.764731 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 358/697] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:45:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 10 12:45:16.787191 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc60e089-2cb8-4620-b904-65007d0fcfba None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:16.915048 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc60e089-2cb8-4620-b904-65007d0fcfba None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:16.922560 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:16.954135 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26f1d90-e9e4-4d8f-a9a3-13eceeb39330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:16.954875 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26f1d90-e9e4-4d8f-a9a3-13eceeb39330 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:16.955163 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26f1d90-e9e4-4d8f-a9a3-13eceeb39330 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:16.955449 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26f1d90-e9e4-4d8f-a9a3-13eceeb39330 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:16.956147 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26f1d90-e9e4-4d8f-a9a3-13eceeb39330 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:16.957511 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26f1d90-e9e4-4d8f-a9a3-13eceeb39330 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:16.971354 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-c26f1d90-e9e4-4d8f-a9a3-13eceeb39330 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 10 12:45:16.971354 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:16.971354 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 10 12:45:16.971354 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 10 12:45:16.971354 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 10 12:45:16.971354 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 10 12:45:16.971354 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 10 12:45:16.971354 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 10 12:45:16.971354 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 10 12:45:16.971354 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:16.971354 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 10 12:45:16.971354 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:16.971354 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:16.971354 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:16.971354 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:16.971354 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:16.972416 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:16.972416 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:16.972416 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:16.972416 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:16.972416 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:16.972416 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:16.972416 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:16.972416 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 10 12:45:16.972416 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 10 12:45:16.972416 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 10 12:45:16.972416 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 10 12:45:16.972416 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:16.972416 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:16.972416 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:16.972416 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:16.972416 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:16.973275 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:16.973275 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:16.973275 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:16.973275 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:16.973275 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:16.973275 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:16.973275 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:16.973275 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:16.973275 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:16.973275 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:16.973275 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:16.973275 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 10 12:45:16.973275 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 10 12:45:16.973275 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 10 12:45:16.973275 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:16.973275 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 340/698] 10.222.0.27 () {56 vars in 1129 bytes} [Fri Jun 10 12:45:16 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:16.978600 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bdad24-8df9-4f04-906a-acb4a6c2f858 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:17.078195 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce013a0b-8df5-4864-b0b9-7f7d8b7d2f88 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:17.078549 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce013a0b-8df5-4864-b0b9-7f7d8b7d2f88 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:17.078776 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce013a0b-8df5-4864-b0b9-7f7d8b7d2f88 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:17.080172 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 341/699] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:45:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:45:17.088364 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edfafa5-0368-416d-b2fb-2bc453c67fed None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:17.088654 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edfafa5-0368-416d-b2fb-2bc453c67fed None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:17.088937 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edfafa5-0368-416d-b2fb-2bc453c67fed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:17.122457 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-3edfafa5-0368-416d-b2fb-2bc453c67fed None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:45:17.123664 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3edfafa5-0368-416d-b2fb-2bc453c67fed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:45:17.149113 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bdad24-8df9-4f04-906a-acb4a6c2f858 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:17.213937 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dce261c-3309-41c5-a32e-10282c468ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:17.214520 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dce261c-3309-41c5-a32e-10282c468ede None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:17.214800 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dce261c-3309-41c5-a32e-10282c468ede None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:17.215075 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dce261c-3309-41c5-a32e-10282c468ede None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:17.215785 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dce261c-3309-41c5-a32e-10282c468ede None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:17.217277 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dce261c-3309-41c5-a32e-10282c468ede None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:17.218646 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dce261c-3309-41c5-a32e-10282c468ede None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:17.225018 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 342/700] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:45:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 10 12:45:17.228661 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 359/701] 10.222.0.27 () {56 vars in 1124 bytes} [Fri Jun 10 12:45:16 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:17.233854 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edfafa5-0368-416d-b2fb-2bc453c67fed None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:17.249696 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dce261c-3309-41c5-a32e-10282c468ede None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:17.254865 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:17.408468 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dce261c-3309-41c5-a32e-10282c468ede None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:17.446671 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f320c3-0e3c-47b9-b8e7-edcdc71dd352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:17.447463 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f320c3-0e3c-47b9-b8e7-edcdc71dd352 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:17.447747 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f320c3-0e3c-47b9-b8e7-edcdc71dd352 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:17.448065 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f320c3-0e3c-47b9-b8e7-edcdc71dd352 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:17.449119 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:17.457987 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edfafa5-0368-416d-b2fb-2bc453c67fed None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:17.460088 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f320c3-0e3c-47b9-b8e7-edcdc71dd352 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:45:17.468536 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-d9f320c3-0e3c-47b9-b8e7-edcdc71dd352 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 10 12:45:17.468536 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:17.468536 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:17.468536 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:17.468536 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:17.468536 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:17.468536 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:17.468536 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:17.468536 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:17.468536 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:17.468536 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:17.468536 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:17.468536 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:45:17.468536 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:45:17.468536 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:45:17.468536 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:45:17.469565 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:45:17.469565 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:45:17.469565 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:45:17.469565 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:45:17.469565 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:17.469565 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:17.469565 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:17.469565 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:17.469565 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:17.469565 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:17.469565 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:17.469565 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:17.469565 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:17.469565 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:17.469565 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:17.469565 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:17.470450 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:17.470450 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:17.470450 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:17.470450 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:17.470450 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:45:17.470450 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:45:17.470450 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:45:17.470450 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:45:17.470450 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:45:17.470450 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:45:17.470450 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 10 12:45:17.470450 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:17.470450 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 360/702] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Jun 10 12:45:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 10 12:45:17.477258 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f320c3-0e3c-47b9-b8e7-edcdc71dd352 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:17.546334 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948f02fb-dd26-463a-ae11-9abcd1df72cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:17.547012 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948f02fb-dd26-463a-ae11-9abcd1df72cf None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:17.547291 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948f02fb-dd26-463a-ae11-9abcd1df72cf None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:17.547563 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948f02fb-dd26-463a-ae11-9abcd1df72cf None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:17.548312 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948f02fb-dd26-463a-ae11-9abcd1df72cf None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:17.548614 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948f02fb-dd26-463a-ae11-9abcd1df72cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:17.549980 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948f02fb-dd26-463a-ae11-9abcd1df72cf None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:17.559403 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:17.564120 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 343/703] 10.222.0.27 () {56 vars in 1124 bytes} [Fri Jun 10 12:45:17 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:17.622150 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f320c3-0e3c-47b9-b8e7-edcdc71dd352 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:17.666173 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6e9fa7-5547-4c8e-ad0a-5fe07b7a8627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:17.666868 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6e9fa7-5547-4c8e-ad0a-5fe07b7a8627 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:17.667152 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6e9fa7-5547-4c8e-ad0a-5fe07b7a8627 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:17.667430 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6e9fa7-5547-4c8e-ad0a-5fe07b7a8627 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:17.668226 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6e9fa7-5547-4c8e-ad0a-5fe07b7a8627 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:17.668531 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6e9fa7-5547-4c8e-ad0a-5fe07b7a8627 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:17.669947 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6e9fa7-5547-4c8e-ad0a-5fe07b7a8627 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:17.685057 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 361/704] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Jun 10 12:45:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 12:45:17.690824 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948f02fb-dd26-463a-ae11-9abcd1df72cf None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:17.812689 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948f02fb-dd26-463a-ae11-9abcd1df72cf None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:17.851979 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37e4b1f-caf1-4713-9335-7059671b4ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:17.852840 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37e4b1f-caf1-4713-9335-7059671b4ad7 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:17.853125 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37e4b1f-caf1-4713-9335-7059671b4ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:17.853402 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37e4b1f-caf1-4713-9335-7059671b4ad7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:17.865411 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37e4b1f-caf1-4713-9335-7059671b4ad7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:17.866956 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37e4b1f-caf1-4713-9335-7059671b4ad7 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:17.884571 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-b37e4b1f-caf1-4713-9335-7059671b4ad7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:17.884571 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:17.884571 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:17.884571 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:17.884571 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:17.884571 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:17.884571 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:17.884571 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:17.884571 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:17.884571 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:17.884571 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:17.884571 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:17.884571 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 10 12:45:17.884571 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 10 12:45:17.884571 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 10 12:45:17.886431 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 10 12:45:17.886431 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:17.886431 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:17.886431 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:17.886431 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:17.886431 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:17.886431 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:17.886431 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:17.886431 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:17.886431 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:17.886431 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:17.886431 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:17.886431 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:17.886431 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:17.886431 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:17.886431 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:17.886431 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:17.887667 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 10 12:45:17.887667 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 10 12:45:17.887667 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:17.887667 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:17.887667 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 344/705] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:45:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:17.891722 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6e9fa7-5547-4c8e-ad0a-5fe07b7a8627 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:18.024171 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6e9fa7-5547-4c8e-ad0a-5fe07b7a8627 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:18.057115 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:18.078780 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4fb16b-5b6e-48f6-9c22-5c0050f9ad16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:18.078780 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4fb16b-5b6e-48f6-9c22-5c0050f9ad16 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:18.078780 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4fb16b-5b6e-48f6-9c22-5c0050f9ad16 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:18.078780 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4fb16b-5b6e-48f6-9c22-5c0050f9ad16 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:18.079502 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4fb16b-5b6e-48f6-9c22-5c0050f9ad16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:18.079502 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4fb16b-5b6e-48f6-9c22-5c0050f9ad16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:18.079502 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4fb16b-5b6e-48f6-9c22-5c0050f9ad16 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:18.091147 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 362/706] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Jun 10 12:45:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 12:45:18.096282 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37e4b1f-caf1-4713-9335-7059671b4ad7 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:18.119855 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:18.227889 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37e4b1f-caf1-4713-9335-7059671b4ad7 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:18.278149 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede2e84a-da20-420e-bd1b-ff0f89a1fa29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:18.278971 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede2e84a-da20-420e-bd1b-ff0f89a1fa29 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:18.279254 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede2e84a-da20-420e-bd1b-ff0f89a1fa29 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:18.279526 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede2e84a-da20-420e-bd1b-ff0f89a1fa29 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:18.288200 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede2e84a-da20-420e-bd1b-ff0f89a1fa29 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:18.289671 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede2e84a-da20-420e-bd1b-ff0f89a1fa29 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:18.303693 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-ede2e84a-da20-420e-bd1b-ff0f89a1fa29 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 10 12:45:18.303693 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:18.303693 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:18.303693 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:18.303693 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:18.303693 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:18.303693 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:18.303693 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:18.303693 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:18.303693 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:18.303693 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:18.303693 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:18.303693 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 10 12:45:18.303693 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 10 12:45:18.303693 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 10 12:45:18.303693 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 10 12:45:18.304725 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:18.304725 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:18.304725 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 10 12:45:18.304725 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 10 12:45:18.304725 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 10 12:45:18.304725 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 10 12:45:18.304725 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:18.304725 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:18.304725 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:18.304725 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:18.304725 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:18.304725 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:18.304725 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:18.304725 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:18.304725 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:18.304725 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:18.304725 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:18.304725 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:18.305705 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:18.305705 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:18.305705 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 10 12:45:18.305705 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 10 12:45:18.305705 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 10 12:45:18.305705 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 10 12:45:18.305705 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 10 12:45:18.305705 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 10 12:45:18.305705 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 10 12:45:18.305705 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:18.305705 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 345/707] 10.222.0.27 () {56 vars in 1093 bytes} [Fri Jun 10 12:45:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:18.311460 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4fb16b-5b6e-48f6-9c22-5c0050f9ad16 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:18.311764 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f27b477-c7f8-4c65-b110-3127e75969d5 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:18.312079 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f27b477-c7f8-4c65-b110-3127e75969d5 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:18.312357 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f27b477-c7f8-4c65-b110-3127e75969d5 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:18.313945 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 346/708] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:45:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:45:18.318769 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a6e233-e515-4ef7-aa0d-6f8514165142 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:18.319054 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a6e233-e515-4ef7-aa0d-6f8514165142 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:18.319334 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a6e233-e515-4ef7-aa0d-6f8514165142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:18.345603 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-80a6e233-e515-4ef7-aa0d-6f8514165142 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:45:18.346759 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-80a6e233-e515-4ef7-aa0d-6f8514165142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:45:18.449417 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4fb16b-5b6e-48f6-9c22-5c0050f9ad16 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:18.477912 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 347/709] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:45:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 10 12:45:18.479956 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df874991-2ee8-4ff2-ba95-848cad038ed3 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:18.480316 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df874991-2ee8-4ff2-ba95-848cad038ed3 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:18.480816 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df874991-2ee8-4ff2-ba95-848cad038ed3 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:18.482939 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 348/710] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:45:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:45:18.486241 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4296f1-78a3-43c3-a323-470f353f26f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:18.487165 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4296f1-78a3-43c3-a323-470f353f26f8 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:18.487165 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4296f1-78a3-43c3-a323-470f353f26f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:18.487463 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4296f1-78a3-43c3-a323-470f353f26f8 None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:18.488330 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4296f1-78a3-43c3-a323-470f353f26f8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:18.488643 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4296f1-78a3-43c3-a323-470f353f26f8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:18.489013 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4c2105-d0ba-4165-9695-1d671c573ee8 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:18.489298 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4c2105-d0ba-4165-9695-1d671c573ee8 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:18.489574 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4c2105-d0ba-4165-9695-1d671c573ee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:18.490064 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4296f1-78a3-43c3-a323-470f353f26f8 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:18.511328 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:18.517743 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-bb4c2105-d0ba-4165-9695-1d671c573ee8 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:45:18.519155 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-bb4c2105-d0ba-4165-9695-1d671c573ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:45:18.523085 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 363/711] 10.222.0.27 () {56 vars in 1133 bytes} [Fri Jun 10 12:45:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:18.526021 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4296f1-78a3-43c3-a323-470f353f26f8 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:18.581641 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:18.600201 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 349/712] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:45:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 10 12:45:18.601459 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4c2105-d0ba-4165-9695-1d671c573ee8 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:18.648500 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4296f1-78a3-43c3-a323-470f353f26f8 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:18.696991 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef64e5ee-d2ae-4dd1-9aea-11d54c69b97b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:18.697801 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef64e5ee-d2ae-4dd1-9aea-11d54c69b97b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:18.698089 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef64e5ee-d2ae-4dd1-9aea-11d54c69b97b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:18.698368 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef64e5ee-d2ae-4dd1-9aea-11d54c69b97b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:18.712275 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef64e5ee-d2ae-4dd1-9aea-11d54c69b97b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:45:18.722867 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-ef64e5ee-d2ae-4dd1-9aea-11d54c69b97b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 10 12:45:18.722867 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:18.722867 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:18.722867 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:18.722867 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:18.722867 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:18.722867 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:18.722867 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:18.722867 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:18.722867 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:18.722867 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:18.722867 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:18.722867 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:45:18.722867 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:45:18.722867 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:45:18.722867 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:45:18.723879 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:45:18.723879 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:45:18.723879 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:45:18.723879 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:45:18.723879 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:18.723879 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:18.723879 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:18.723879 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:18.723879 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:18.723879 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:18.723879 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:18.723879 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:18.723879 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:18.723879 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:18.723879 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:18.723879 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:18.724782 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:18.724782 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:18.724782 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:18.724782 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:18.724782 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:45:18.724782 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:45:18.724782 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:45:18.724782 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:45:18.724782 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:45:18.724782 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:45:18.724782 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 10 12:45:18.724782 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:18.724782 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 364/713] 10.222.0.27 () {56 vars in 1102 bytes} [Fri Jun 10 12:45:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:18.725519 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef64e5ee-d2ae-4dd1-9aea-11d54c69b97b None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:18.731700 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4c2105-d0ba-4165-9695-1d671c573ee8 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:18.777128 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763bb5cd-758e-440d-8ff2-6def6abae07b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:18.777797 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763bb5cd-758e-440d-8ff2-6def6abae07b None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:18.778087 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763bb5cd-758e-440d-8ff2-6def6abae07b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:18.778372 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763bb5cd-758e-440d-8ff2-6def6abae07b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:18.782839 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:18.835457 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef64e5ee-d2ae-4dd1-9aea-11d54c69b97b None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:18.857558 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:18.867915 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763bb5cd-758e-440d-8ff2-6def6abae07b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:18.867915 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763bb5cd-758e-440d-8ff2-6def6abae07b None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:18.876358 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b160f895-1076-4516-9961-a14958094cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:18.881868 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b160f895-1076-4516-9961-a14958094cfc None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:18.881868 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b160f895-1076-4516-9961-a14958094cfc None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:18.881868 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b160f895-1076-4516-9961-a14958094cfc None admin] PATH_INFO: `/v3/services/network` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:18.881868 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b160f895-1076-4516-9961-a14958094cfc None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:18.881868 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b160f895-1076-4516-9961-a14958094cfc None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:18.894937 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-b160f895-1076-4516-9961-a14958094cfc None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 10 12:45:18.894937 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:18.894937 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 10 12:45:18.894937 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 10 12:45:18.894937 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 10 12:45:18.894937 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 10 12:45:18.894937 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 10 12:45:18.894937 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 10 12:45:18.894937 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 10 12:45:18.894937 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:18.894937 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 10 12:45:18.894937 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:18.894937 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:18.894937 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:18.894937 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:18.894937 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:18.894937 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:18.896334 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:18.896334 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:18.896334 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:18.896334 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:18.896334 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:18.896334 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:18.896334 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 10 12:45:18.896334 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 10 12:45:18.896334 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 10 12:45:18.896334 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 10 12:45:18.896334 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:18.896334 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:18.896334 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:18.896334 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:18.896334 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:18.896334 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:18.897267 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:18.897267 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:18.897267 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:18.897267 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:18.897267 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:18.897267 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:18.897267 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:18.897267 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:18.897267 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:18.897267 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:18.897267 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 10 12:45:18.897267 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 10 12:45:18.897267 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 10 12:45:18.897267 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:18.901283 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 365/714] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Jun 10 12:45:18 2022] GET /identity/v3/services/network => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:18.902357 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b160f895-1076-4516-9961-a14958094cfc None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:18.985490 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 350/715] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Jun 10 12:45:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 10 12:45:18.986511 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763bb5cd-758e-440d-8ff2-6def6abae07b None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:19.009139 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b160f895-1076-4516-9961-a14958094cfc None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:19.023764 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:19.045740 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e06e40-6e6c-47ea-8028-cb3c69a13a08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:19.046550 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e06e40-6e6c-47ea-8028-cb3c69a13a08 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:19.046941 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e06e40-6e6c-47ea-8028-cb3c69a13a08 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:19.047319 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e06e40-6e6c-47ea-8028-cb3c69a13a08 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:19.048229 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e06e40-6e6c-47ea-8028-cb3c69a13a08 None admin] RBAC: Adding query filter params (name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:19.048640 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e06e40-6e6c-47ea-8028-cb3c69a13a08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:19.050123 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e06e40-6e6c-47ea-8028-cb3c69a13a08 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:19.060430 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 366/716] 10.222.0.27 () {56 vars in 1103 bytes} [Fri Jun 10 12:45:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:19.061686 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e06e40-6e6c-47ea-8028-cb3c69a13a08 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:19.092991 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763bb5cd-758e-440d-8ff2-6def6abae07b None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:19.093819 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:19.128242 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8afd157-3203-4dec-bee6-84c7b9655f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:19.128838 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8afd157-3203-4dec-bee6-84c7b9655f89 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:19.129123 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8afd157-3203-4dec-bee6-84c7b9655f89 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:19.129399 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8afd157-3203-4dec-bee6-84c7b9655f89 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:19.130110 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8afd157-3203-4dec-bee6-84c7b9655f89 None admin] RBAC: Adding query filter params (name=network) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:19.130417 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8afd157-3203-4dec-bee6-84c7b9655f89 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:19.131782 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8afd157-3203-4dec-bee6-84c7b9655f89 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:19.136813 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 351/717] 10.222.0.27 () {56 vars in 1112 bytes} [Fri Jun 10 12:45:18 2022] GET /identity/v3/services?name=network => generated 124 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:19.137679 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8afd157-3203-4dec-bee6-84c7b9655f89 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:19.183699 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e06e40-6e6c-47ea-8028-cb3c69a13a08 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:19.222544 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f38a658-39b4-4403-9e98-cec5ddf8b4c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:19.223413 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f38a658-39b4-4403-9e98-cec5ddf8b4c8 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:19.223699 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f38a658-39b4-4403-9e98-cec5ddf8b4c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:19.224002 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f38a658-39b4-4403-9e98-cec5ddf8b4c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:19.236760 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f38a658-39b4-4403-9e98-cec5ddf8b4c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:19.238358 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f38a658-39b4-4403-9e98-cec5ddf8b4c8 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:19.244203 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:19.246958 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8afd157-3203-4dec-bee6-84c7b9655f89 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:19.259884 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-1f38a658-39b4-4403-9e98-cec5ddf8b4c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:19.259884 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:19.259884 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:19.259884 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:19.259884 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:19.259884 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:19.259884 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:19.259884 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:19.259884 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:19.259884 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:19.259884 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:19.259884 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:19.259884 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 10 12:45:19.259884 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 10 12:45:19.259884 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 10 12:45:19.260990 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 10 12:45:19.260990 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:19.260990 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:19.260990 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:19.260990 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:19.260990 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:19.260990 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:19.260990 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:19.260990 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:19.260990 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:19.260990 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:19.260990 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:19.260990 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:19.260990 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:19.260990 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:19.260990 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:19.260990 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:19.262074 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 10 12:45:19.262074 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 10 12:45:19.262074 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:19.262074 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:19.268165 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 367/718] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:45:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:19.269171 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f38a658-39b4-4403-9e98-cec5ddf8b4c8 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:19.283897 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa08113c-bebe-44da-be4d-ccd45a87b961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:19.284680 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa08113c-bebe-44da-be4d-ccd45a87b961 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:19.284967 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa08113c-bebe-44da-be4d-ccd45a87b961 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:19.285247 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa08113c-bebe-44da-be4d-ccd45a87b961 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:19.295597 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa08113c-bebe-44da-be4d-ccd45a87b961 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:19.297187 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa08113c-bebe-44da-be4d-ccd45a87b961 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:19.314737 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-aa08113c-bebe-44da-be4d-ccd45a87b961 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:19.314737 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:19.314737 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:19.314737 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:19.314737 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:19.314737 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:19.314737 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:19.314737 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:19.314737 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:19.314737 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:19.314737 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:19.314737 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:19.314737 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 10 12:45:19.314737 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 10 12:45:19.314737 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 10 12:45:19.315791 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 10 12:45:19.315791 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:19.315791 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:19.315791 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:19.315791 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:19.315791 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:19.315791 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:19.315791 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:19.315791 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:19.315791 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:19.315791 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:19.315791 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:19.315791 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:19.315791 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:19.315791 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:19.315791 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:19.315791 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:19.316758 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 10 12:45:19.316758 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 10 12:45:19.316758 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:19.316758 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:19.316758 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 352/719] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:45:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:19.317044 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa08113c-bebe-44da-be4d-ccd45a87b961 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:19.362085 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:19.401255 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f38a658-39b4-4403-9e98-cec5ddf8b4c8 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:19.416721 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7aff99-f21a-4d56-945b-c0fe7d162557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:19.417436 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7aff99-f21a-4d56-945b-c0fe7d162557 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:19.417720 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7aff99-f21a-4d56-945b-c0fe7d162557 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:19.418191 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7aff99-f21a-4d56-945b-c0fe7d162557 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:19.419243 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7aff99-f21a-4d56-945b-c0fe7d162557 None admin] RBAC: Adding query filter params (type=network) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:19.420144 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7aff99-f21a-4d56-945b-c0fe7d162557 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:19.421512 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7aff99-f21a-4d56-945b-c0fe7d162557 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:19.421771 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa08113c-bebe-44da-be4d-ccd45a87b961 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:19.427523 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 368/720] 10.222.0.27 () {56 vars in 1112 bytes} [Fri Jun 10 12:45:19 2022] GET /identity/v3/services?type=network => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:19.428593 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72034e6e-4b13-423b-af57-a70ecd47fdca None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:19.428880 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72034e6e-4b13-423b-af57-a70ecd47fdca None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:19.429694 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72034e6e-4b13-423b-af57-a70ecd47fdca None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:19.431406 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 369/721] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:45:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:45:19.432035 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72034e6e-4b13-423b-af57-a70ecd47fdca None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:19.471042 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9a4e4e-339a-41f8-92e4-330b16ca1d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:19.472132 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9a4e4e-339a-41f8-92e4-330b16ca1d35 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:19.472614 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9a4e4e-339a-41f8-92e4-330b16ca1d35 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:19.473354 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9a4e4e-339a-41f8-92e4-330b16ca1d35 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:19.475307 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9a4e4e-339a-41f8-92e4-330b16ca1d35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:19.476399 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9a4e4e-339a-41f8-92e4-330b16ca1d35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:19.478007 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9a4e4e-339a-41f8-92e4-330b16ca1d35 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:19.501497 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 353/722] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Jun 10 12:45:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:19.501497 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7923f807-21e4-4cd3-a157-749b81ec4a24 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:19.501497 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7923f807-21e4-4cd3-a157-749b81ec4a24 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:19.501497 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7923f807-21e4-4cd3-a157-749b81ec4a24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:19.519087 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-7923f807-21e4-4cd3-a157-749b81ec4a24 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:45:19.520426 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-7923f807-21e4-4cd3-a157-749b81ec4a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:45:19.530234 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:19.575029 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72034e6e-4b13-423b-af57-a70ecd47fdca None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:19.599034 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 354/723] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:45:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 10 12:45:19.603963 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7923f807-21e4-4cd3-a157-749b81ec4a24 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:19.618867 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5837884d-6be4-4393-8d06-4e10afe77de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:19.620265 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5837884d-6be4-4393-8d06-4e10afe77de8 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:19.621001 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5837884d-6be4-4393-8d06-4e10afe77de8 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:19.621500 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5837884d-6be4-4393-8d06-4e10afe77de8 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:19.622467 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5837884d-6be4-4393-8d06-4e10afe77de8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:19.623162 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5837884d-6be4-4393-8d06-4e10afe77de8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:19.625533 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5837884d-6be4-4393-8d06-4e10afe77de8 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:19.641913 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 370/724] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Jun 10 12:45:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:19.643320 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5837884d-6be4-4393-8d06-4e10afe77de8 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:19.650291 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:19.741158 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7923f807-21e4-4cd3-a157-749b81ec4a24 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:19.761388 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5837884d-6be4-4393-8d06-4e10afe77de8 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:19.789253 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba10815-ed5d-4e7e-a5f3-a20438a907c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:19.790733 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba10815-ed5d-4e7e-a5f3-a20438a907c8 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:19.791445 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba10815-ed5d-4e7e-a5f3-a20438a907c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:19.791884 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba10815-ed5d-4e7e-a5f3-a20438a907c8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:19.798398 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4432460d-65cc-41c0-a77c-091bf0bc27a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:19.799329 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4432460d-65cc-41c0-a77c-091bf0bc27a7 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:19.799714 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4432460d-65cc-41c0-a77c-091bf0bc27a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:19.800145 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4432460d-65cc-41c0-a77c-091bf0bc27a7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:19.801004 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba10815-ed5d-4e7e-a5f3-a20438a907c8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:19.802904 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba10815-ed5d-4e7e-a5f3-a20438a907c8 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:19.811976 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4432460d-65cc-41c0-a77c-091bf0bc27a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:45:19.819709 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-1ba10815-ed5d-4e7e-a5f3-a20438a907c8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 10 12:45:19.819709 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:19.819709 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:19.819709 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:19.819709 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:19.819709 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:19.819709 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:19.819709 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:19.819709 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:19.819709 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:19.819709 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:19.819709 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:19.819709 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 10 12:45:19.819709 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 10 12:45:19.819709 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 10 12:45:19.820816 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 10 12:45:19.820816 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:19.820816 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:19.820816 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:19.820816 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:19.820816 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:19.820816 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:19.820816 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:19.820816 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:19.820816 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:19.820816 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:19.820816 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:19.820816 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:19.820816 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:19.820816 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:19.820816 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:19.820816 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:19.822022 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 10 12:45:19.822022 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 10 12:45:19.822022 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 10 12:45:19.822022 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 10 12:45:19.822022 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 10 12:45:19.822022 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 10 12:45:19.822022 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 10 12:45:19.822022 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:19.822570 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-4432460d-65cc-41c0-a77c-091bf0bc27a7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 10 12:45:19.822570 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:19.822570 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:19.822570 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:19.822570 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:19.822570 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:19.822570 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:19.822570 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:19.822570 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:19.822570 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:19.822570 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:19.822570 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:19.822570 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:45:19.822570 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:45:19.822570 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:45:19.822570 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:45:19.823588 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:45:19.823588 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:45:19.823588 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:45:19.823588 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:45:19.823588 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:19.823588 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:19.823588 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:19.823588 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:19.823588 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:19.823588 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:19.823588 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:19.823588 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:19.823588 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:19.823588 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:19.823588 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:19.823588 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:19.824748 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:19.824748 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:19.824748 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:19.824748 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:19.824748 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:45:19.824748 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:45:19.824748 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:45:19.824748 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:45:19.824748 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:45:19.824748 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:45:19.824748 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 10 12:45:19.824748 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:19.831244 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 371/725] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Jun 10 12:45:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:19.831244 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4432460d-65cc-41c0-a77c-091bf0bc27a7 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:19.832144 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 355/726] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Jun 10 12:45:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:19.834110 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba10815-ed5d-4e7e-a5f3-a20438a907c8 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:19.837602 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:19.910013 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:19.954332 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4432460d-65cc-41c0-a77c-091bf0bc27a7 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:19.962714 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba10815-ed5d-4e7e-a5f3-a20438a907c8 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:19.989728 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f44bd5-2e59-4099-ac59-17b29fae0176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:19.990555 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f44bd5-2e59-4099-ac59-17b29fae0176 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:19.990844 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f44bd5-2e59-4099-ac59-17b29fae0176 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:19.991125 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f44bd5-2e59-4099-ac59-17b29fae0176 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:19.999199 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f44bd5-2e59-4099-ac59-17b29fae0176 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:20.000723 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f44bd5-2e59-4099-ac59-17b29fae0176 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:20.001683 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90762bc2-8038-4cca-9903-a4d14975a303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:20.002671 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90762bc2-8038-4cca-9903-a4d14975a303 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:20.003276 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90762bc2-8038-4cca-9903-a4d14975a303 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:20.003760 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90762bc2-8038-4cca-9903-a4d14975a303 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:20.004726 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90762bc2-8038-4cca-9903-a4d14975a303 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:20.005247 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90762bc2-8038-4cca-9903-a4d14975a303 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:20.006818 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90762bc2-8038-4cca-9903-a4d14975a303 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:20.015901 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-92f44bd5-2e59-4099-ac59-17b29fae0176 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 10 12:45:20.015901 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:20.015901 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:20.015901 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:20.015901 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:20.015901 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:20.015901 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:20.015901 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:20.015901 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:20.015901 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:20.015901 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:20.015901 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:20.015901 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 10 12:45:20.015901 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 10 12:45:20.015901 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 10 12:45:20.015901 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 10 12:45:20.016855 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:20.016855 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:20.016855 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 10 12:45:20.016855 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 10 12:45:20.016855 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 10 12:45:20.016855 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 10 12:45:20.016855 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:20.016855 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:20.016855 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:20.016855 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:20.016855 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:20.016855 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:20.016855 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:20.016855 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:20.016855 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:20.016855 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:20.016855 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:20.016855 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:20.017815 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:20.017815 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:20.017815 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 10 12:45:20.017815 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 10 12:45:20.017815 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 10 12:45:20.017815 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 10 12:45:20.017815 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 10 12:45:20.017815 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 10 12:45:20.017815 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 10 12:45:20.017815 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:20.017815 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 372/728] 10.222.0.27 () {56 vars in 1117 bytes} [Fri Jun 10 12:45:19 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:20.018417 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f44bd5-2e59-4099-ac59-17b29fae0176 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:20.019106 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 356/728] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Jun 10 12:45:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:20.028948 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90762bc2-8038-4cca-9903-a4d14975a303 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:20.108265 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:20.145293 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f44bd5-2e59-4099-ac59-17b29fae0176 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:20.156743 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90762bc2-8038-4cca-9903-a4d14975a303 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:20.172536 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:20.183348 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d92666-5526-4eeb-8a7d-570b2b761e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:20.184153 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d92666-5526-4eeb-8a7d-570b2b761e19 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:20.184436 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d92666-5526-4eeb-8a7d-570b2b761e19 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:20.184714 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d92666-5526-4eeb-8a7d-570b2b761e19 None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:20.185803 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d92666-5526-4eeb-8a7d-570b2b761e19 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:20.185803 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d92666-5526-4eeb-8a7d-570b2b761e19 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:20.187285 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d92666-5526-4eeb-8a7d-570b2b761e19 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:20.200617 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 373/729] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Jun 10 12:45:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:20.201437 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d92666-5526-4eeb-8a7d-570b2b761e19 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:20.206563 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb91b68-44fe-4544-b089-ac3e5b3c0343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:20.208365 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb91b68-44fe-4544-b089-ac3e5b3c0343 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:20.208665 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb91b68-44fe-4544-b089-ac3e5b3c0343 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:20.208927 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb91b68-44fe-4544-b089-ac3e5b3c0343 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:20.220878 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb91b68-44fe-4544-b089-ac3e5b3c0343 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:20.222314 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb91b68-44fe-4544-b089-ac3e5b3c0343 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:20.240085 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-beb91b68-44fe-4544-b089-ac3e5b3c0343 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:20.240085 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:20.240085 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:20.240085 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:20.240085 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:20.240085 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:20.240085 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:20.240085 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:20.240085 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:20.240085 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:20.240085 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:20.240085 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:20.240085 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 10 12:45:20.240085 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 10 12:45:20.240085 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 10 12:45:20.241009 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 10 12:45:20.241009 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:20.241009 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:20.241009 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:20.241009 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:20.241009 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:20.241009 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:20.241009 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:20.241009 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:20.241009 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:20.241009 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:20.241009 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:20.241009 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:20.241009 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:20.241009 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:20.241009 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:20.241009 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:20.241965 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 10 12:45:20.241965 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 10 12:45:20.241965 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:20.241965 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:20.241965 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 357/730] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:45:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:20.243223 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de49984-5829-4656-a1d2-df60e8605669 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:20.243829 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de49984-5829-4656-a1d2-df60e8605669 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:20.244144 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de49984-5829-4656-a1d2-df60e8605669 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:20.245817 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 358/731] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:45:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:45:20.246221 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de49984-5829-4656-a1d2-df60e8605669 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:20.318551 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d92666-5526-4eeb-8a7d-570b2b761e19 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:20.352024 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de49984-5829-4656-a1d2-df60e8605669 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:20.357988 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25493539-dd16-45d7-be42-345a0d6c76b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:20.358670 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25493539-dd16-45d7-be42-345a0d6c76b6 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:20.358957 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25493539-dd16-45d7-be42-345a0d6c76b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:20.359234 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25493539-dd16-45d7-be42-345a0d6c76b6 None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:20.360130 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25493539-dd16-45d7-be42-345a0d6c76b6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:20.360442 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25493539-dd16-45d7-be42-345a0d6c76b6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:20.362387 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25493539-dd16-45d7-be42-345a0d6c76b6 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:20.370771 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:20.382908 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 374/732] 10.222.0.27 () {56 vars in 1149 bytes} [Fri Jun 10 12:45:20 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:20.382908 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25493539-dd16-45d7-be42-345a0d6c76b6 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:20.394459 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e75ae30-49c6-4f50-af16-ffdbf32d77c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:20.395121 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e75ae30-49c6-4f50-af16-ffdbf32d77c4 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:20.395405 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e75ae30-49c6-4f50-af16-ffdbf32d77c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:20.395683 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e75ae30-49c6-4f50-af16-ffdbf32d77c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:20.396563 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e75ae30-49c6-4f50-af16-ffdbf32d77c4 None admin] RBAC: Adding query filter params (role.id=b43d5a8247e244869c2c70f986f42e8a, scope.project.id=73ea5b23a2d34544ab493f4c5e4034b8, user.id=8bd6caa461e84100a3d5fe5af5d80f3e) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:20.396875 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e75ae30-49c6-4f50-af16-ffdbf32d77c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:20.398877 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e75ae30-49c6-4f50-af16-ffdbf32d77c4 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:20.405351 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 359/733] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Jun 10 12:45:20 2022] GET /identity/v3/role_assignments?user.id=8bd6caa461e84100a3d5fe5af5d80f3e&scope.project.id=73ea5b23a2d34544ab493f4c5e4034b8&role.id=b43d5a8247e244869c2c70f986f42e8a => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:20.406351 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6e297f-9d9a-4ea3-bcb6-96723d011f47 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:20.406641 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6e297f-9d9a-4ea3-bcb6-96723d011f47 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:20.406919 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6e297f-9d9a-4ea3-bcb6-96723d011f47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:20.425119 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-2f6e297f-9d9a-4ea3-bcb6-96723d011f47 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:45:20.426359 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2f6e297f-9d9a-4ea3-bcb6-96723d011f47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:45:20.448318 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:20.512446 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 360/734] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:45:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 10 12:45:20.514035 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6e297f-9d9a-4ea3-bcb6-96723d011f47 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:20.542264 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25493539-dd16-45d7-be42-345a0d6c76b6 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:20.588364 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a08fd0-52ff-4518-924e-04bb82fd806f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:20.588972 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a08fd0-52ff-4518-924e-04bb82fd806f None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:20.589261 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a08fd0-52ff-4518-924e-04bb82fd806f None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:20.589549 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a08fd0-52ff-4518-924e-04bb82fd806f None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:20.590539 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a08fd0-52ff-4518-924e-04bb82fd806f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:20.590655 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a08fd0-52ff-4518-924e-04bb82fd806f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:20.592059 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a08fd0-52ff-4518-924e-04bb82fd806f None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:20.605111 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 375/735] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Jun 10 12:45:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:20.605944 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a08fd0-52ff-4518-924e-04bb82fd806f None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:20.644456 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6e297f-9d9a-4ea3-bcb6-96723d011f47 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:20.655965 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:20.683837 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6de2465-84b0-4a4f-b5b6-df0506c8188c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:20.684680 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6de2465-84b0-4a4f-b5b6-df0506c8188c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:20.685062 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6de2465-84b0-4a4f-b5b6-df0506c8188c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:20.685468 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6de2465-84b0-4a4f-b5b6-df0506c8188c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:20.715949 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a08fd0-52ff-4518-924e-04bb82fd806f None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:20.717932 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:20.755512 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354bba41-7384-4e90-b373-154004f2da6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:20.756358 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6de2465-84b0-4a4f-b5b6-df0506c8188c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:20.756786 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354bba41-7384-4e90-b373-154004f2da6c None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:20.757811 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354bba41-7384-4e90-b373-154004f2da6c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:20.758243 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354bba41-7384-4e90-b373-154004f2da6c None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:20.758740 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6de2465-84b0-4a4f-b5b6-df0506c8188c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:20.759303 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354bba41-7384-4e90-b373-154004f2da6c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:20.760982 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354bba41-7384-4e90-b373-154004f2da6c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:20.837472 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 376/736] 10.222.0.27 () {60 vars in 1141 bytes} [Fri Jun 10 12:45:20 2022] POST /identity/v3/services => generated 251 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 12:45:20.838898 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354bba41-7384-4e90-b373-154004f2da6c None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:20.861018 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 361/737] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Jun 10 12:45:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 10 12:45:20.867782 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6de2465-84b0-4a4f-b5b6-df0506c8188c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:20.879207 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:20.972553 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6de2465-84b0-4a4f-b5b6-df0506c8188c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:20.985275 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354bba41-7384-4e90-b373-154004f2da6c None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:21.010437 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3793f6-7c27-4392-b43f-629c4f29fabe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:21.011238 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3793f6-7c27-4392-b43f-629c4f29fabe None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:21.011519 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3793f6-7c27-4392-b43f-629c4f29fabe None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:21.011797 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3793f6-7c27-4392-b43f-629c4f29fabe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:21.022791 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3793f6-7c27-4392-b43f-629c4f29fabe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:21.024602 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3793f6-7c27-4392-b43f-629c4f29fabe None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:21.027127 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56111aff-e0fc-4362-98ff-df0d1743e677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:21.028385 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56111aff-e0fc-4362-98ff-df0d1743e677 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:21.028882 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56111aff-e0fc-4362-98ff-df0d1743e677 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:21.029363 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56111aff-e0fc-4362-98ff-df0d1743e677 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:21.039030 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56111aff-e0fc-4362-98ff-df0d1743e677 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:21.040907 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56111aff-e0fc-4362-98ff-df0d1743e677 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:21.044596 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-af3793f6-7c27-4392-b43f-629c4f29fabe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:21.044596 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:21.044596 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:21.044596 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:21.044596 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:21.044596 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:21.044596 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:21.044596 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:21.044596 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:21.044596 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:21.044596 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:21.044596 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:21.044596 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 10 12:45:21.044596 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 10 12:45:21.044596 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 10 12:45:21.045548 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 10 12:45:21.045548 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:21.045548 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:21.045548 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:21.045548 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:21.045548 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:21.045548 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:21.045548 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:21.045548 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:21.045548 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:21.045548 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:21.045548 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:21.045548 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:21.045548 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:21.045548 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:21.045548 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:21.045548 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:21.047089 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 10 12:45:21.047089 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 10 12:45:21.047089 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:21.047089 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:21.047089 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 362/738] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:45:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:21.051369 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3793f6-7c27-4392-b43f-629c4f29fabe None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:21.061977 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-56111aff-e0fc-4362-98ff-df0d1743e677 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 10 12:45:21.061977 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:21.061977 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:21.061977 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:21.061977 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:21.061977 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:21.061977 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:21.061977 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:21.061977 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:21.061977 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:21.061977 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:21.061977 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:21.061977 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 10 12:45:21.061977 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 10 12:45:21.061977 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 10 12:45:21.061977 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 10 12:45:21.062985 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:21.062985 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:21.062985 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 10 12:45:21.062985 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 10 12:45:21.062985 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 10 12:45:21.062985 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 10 12:45:21.062985 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:21.062985 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:21.062985 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:21.062985 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:21.062985 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:21.062985 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:21.062985 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:21.062985 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:21.062985 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:21.062985 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:21.062985 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:21.062985 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:21.064018 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:21.064018 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:21.064018 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 10 12:45:21.064018 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 10 12:45:21.064018 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 10 12:45:21.064018 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 10 12:45:21.064018 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 10 12:45:21.064018 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 10 12:45:21.064018 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 10 12:45:21.064018 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:21.066935 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 377/739] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Jun 10 12:45:20 2022] GET /identity/v3/users/swift => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:21.076828 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56111aff-e0fc-4362-98ff-df0d1743e677 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:21.110991 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:21.123256 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:21.162585 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3793f6-7c27-4392-b43f-629c4f29fabe None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:21.202279 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62f3388-0e59-4cfb-a225-8d5537f1b83a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:21.202960 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62f3388-0e59-4cfb-a225-8d5537f1b83a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:21.203321 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62f3388-0e59-4cfb-a225-8d5537f1b83a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:21.203606 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62f3388-0e59-4cfb-a225-8d5537f1b83a None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:21.204371 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62f3388-0e59-4cfb-a225-8d5537f1b83a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:21.204682 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62f3388-0e59-4cfb-a225-8d5537f1b83a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:21.206051 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62f3388-0e59-4cfb-a225-8d5537f1b83a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:21.218842 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 363/740] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Jun 10 12:45:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:21.221487 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56111aff-e0fc-4362-98ff-df0d1743e677 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:21.224150 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62f3388-0e59-4cfb-a225-8d5537f1b83a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:21.258303 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf86bbc3-0485-4c0c-ab74-bbb0bbf8642a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:21.258955 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf86bbc3-0485-4c0c-ab74-bbb0bbf8642a None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:21.259239 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf86bbc3-0485-4c0c-ab74-bbb0bbf8642a None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:21.259519 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf86bbc3-0485-4c0c-ab74-bbb0bbf8642a None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:21.260474 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf86bbc3-0485-4c0c-ab74-bbb0bbf8642a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:21.260793 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf86bbc3-0485-4c0c-ab74-bbb0bbf8642a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:21.262238 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf86bbc3-0485-4c0c-ab74-bbb0bbf8642a None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:21.281652 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 378/741] 10.222.0.27 () {56 vars in 1135 bytes} [Fri Jun 10 12:45:21 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:21.287359 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf86bbc3-0485-4c0c-ab74-bbb0bbf8642a None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:21.337006 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62f3388-0e59-4cfb-a225-8d5537f1b83a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:21.362465 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:21.370964 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:21.376537 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434b9884-8b43-4e82-8466-c56c68b502cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:21.377549 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434b9884-8b43-4e82-8466-c56c68b502cb None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:21.377975 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434b9884-8b43-4e82-8466-c56c68b502cb None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:21.383830 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434b9884-8b43-4e82-8466-c56c68b502cb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:21.385936 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434b9884-8b43-4e82-8466-c56c68b502cb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:21.387590 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434b9884-8b43-4e82-8466-c56c68b502cb None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:21.404598 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-434b9884-8b43-4e82-8466-c56c68b502cb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 10 12:45:21.404598 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:21.404598 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:21.404598 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:21.404598 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:21.404598 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:21.404598 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:21.404598 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:21.404598 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:21.404598 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:21.404598 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:21.404598 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:21.404598 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 10 12:45:21.404598 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 10 12:45:21.404598 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 10 12:45:21.413106 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 10 12:45:21.413106 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:21.413106 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:21.413106 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:21.413106 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:21.413106 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:21.413106 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:21.413106 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:21.413106 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:21.413106 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:21.413106 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:21.413106 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:21.413106 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:21.413106 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:21.413106 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:21.413106 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:21.413106 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:21.414773 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 10 12:45:21.414773 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 10 12:45:21.414773 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 10 12:45:21.414773 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 10 12:45:21.414773 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 10 12:45:21.414773 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 10 12:45:21.414773 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 10 12:45:21.414773 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:21.414773 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 364/742] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Jun 10 12:45:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 10 12:45:21.416291 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434b9884-8b43-4e82-8466-c56c68b502cb None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:21.473802 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf86bbc3-0485-4c0c-ab74-bbb0bbf8642a None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:21.524489 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434b9884-8b43-4e82-8466-c56c68b502cb None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:21.562336 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe849b8-12bf-4f14-bd6b-8423186e2598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:21.563465 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe849b8-12bf-4f14-bd6b-8423186e2598 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:21.563946 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe849b8-12bf-4f14-bd6b-8423186e2598 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:21.564749 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe849b8-12bf-4f14-bd6b-8423186e2598 None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:21.566009 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe849b8-12bf-4f14-bd6b-8423186e2598 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:21.566484 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe849b8-12bf-4f14-bd6b-8423186e2598 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:21.568130 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe849b8-12bf-4f14-bd6b-8423186e2598 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:21.581545 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 365/743] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Jun 10 12:45:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:21.588109 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe849b8-12bf-4f14-bd6b-8423186e2598 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:21.608704 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f57498c-5631-4486-9592-33e3e398a79b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:21.609396 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f57498c-5631-4486-9592-33e3e398a79b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:21.609684 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f57498c-5631-4486-9592-33e3e398a79b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:21.609963 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f57498c-5631-4486-9592-33e3e398a79b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:21.652486 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:21.703533 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe849b8-12bf-4f14-bd6b-8423186e2598 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:21.750195 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f57498c-5631-4486-9592-33e3e398a79b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:21.750769 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7591c3f6-6243-492c-bcbc-67cba6220a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:21.751675 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f57498c-5631-4486-9592-33e3e398a79b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:21.752390 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7591c3f6-6243-492c-bcbc-67cba6220a0a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:21.752813 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7591c3f6-6243-492c-bcbc-67cba6220a0a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:21.753241 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7591c3f6-6243-492c-bcbc-67cba6220a0a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:21.755664 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7591c3f6-6243-492c-bcbc-67cba6220a0a None admin] RBAC: Adding query filter params (role.id=ea7e8f74523b4322857fb3f4307d171e, scope.project.id=73ea5b23a2d34544ab493f4c5e4034b8, user.id=da66362070694e91bbe2cfbbd161b8a7) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:21.756289 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7591c3f6-6243-492c-bcbc-67cba6220a0a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:21.758404 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7591c3f6-6243-492c-bcbc-67cba6220a0a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:21.766394 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 366/744] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Jun 10 12:45:21 2022] GET /identity/v3/role_assignments?user.id=da66362070694e91bbe2cfbbd161b8a7&scope.project.id=73ea5b23a2d34544ab493f4c5e4034b8&role.id=ea7e8f74523b4322857fb3f4307d171e => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:21.772400 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b897c5e-3069-4ea2-b5f8-f705381ccd8d None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:21.772400 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b897c5e-3069-4ea2-b5f8-f705381ccd8d None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:21.772400 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b897c5e-3069-4ea2-b5f8-f705381ccd8d None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:21.776975 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 367/745] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:45:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:45:21.778350 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e395ffe-2478-4e54-add7-24adc8e315aa None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:21.778638 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e395ffe-2478-4e54-add7-24adc8e315aa None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:21.778914 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e395ffe-2478-4e54-add7-24adc8e315aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:21.812901 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-1e395ffe-2478-4e54-add7-24adc8e315aa None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:45:21.814268 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1e395ffe-2478-4e54-add7-24adc8e315aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:45:21.879357 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:21.928917 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 379/746] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Jun 10 12:45:21 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 642 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 10 12:45:21.934147 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f57498c-5631-4486-9592-33e3e398a79b None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:21.959829 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 368/747] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:45:21 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 10 12:45:21.996955 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e395ffe-2478-4e54-add7-24adc8e315aa None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:22.041392 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:22.110424 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e395ffe-2478-4e54-add7-24adc8e315aa None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:22.146350 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce4d17e-b511-498f-9fcd-17af32ae6fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:22.147015 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce4d17e-b511-498f-9fcd-17af32ae6fae None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:22.147304 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce4d17e-b511-498f-9fcd-17af32ae6fae None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:22.147608 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce4d17e-b511-498f-9fcd-17af32ae6fae None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:22.148397 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce4d17e-b511-498f-9fcd-17af32ae6fae None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:22.149817 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce4d17e-b511-498f-9fcd-17af32ae6fae None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:22.232231 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f57498c-5631-4486-9592-33e3e398a79b None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:22.243815 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 369/748] 10.222.0.27 () {60 vars in 1141 bytes} [Fri Jun 10 12:45:21 2022] POST /identity/v3/services => generated 240 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 12:45:22.281454 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee9c9d3-80ab-4f28-be40-3e9dfcdbba13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:22.282531 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee9c9d3-80ab-4f28-be40-3e9dfcdbba13 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:22.282957 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee9c9d3-80ab-4f28-be40-3e9dfcdbba13 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:22.283374 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee9c9d3-80ab-4f28-be40-3e9dfcdbba13 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:22.296607 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee9c9d3-80ab-4f28-be40-3e9dfcdbba13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:22.298389 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee9c9d3-80ab-4f28-be40-3e9dfcdbba13 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:22.318748 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-9ee9c9d3-80ab-4f28-be40-3e9dfcdbba13 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:22.318748 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:22.318748 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:22.318748 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:22.318748 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:22.318748 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:22.318748 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:22.318748 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:22.318748 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:22.318748 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:22.318748 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:22.318748 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:22.318748 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 10 12:45:22.318748 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 10 12:45:22.318748 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 10 12:45:22.320571 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 10 12:45:22.320571 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:22.320571 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:22.320571 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:22.320571 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:22.320571 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:22.320571 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:22.320571 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:22.320571 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:22.320571 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:22.320571 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:22.320571 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:22.320571 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:22.320571 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:22.320571 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:22.320571 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:22.320571 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:22.321688 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 10 12:45:22.321688 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 10 12:45:22.321688 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:22.321688 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:22.323678 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 380/749] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:45:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 390 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 10 12:45:22.333028 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce4d17e-b511-498f-9fcd-17af32ae6fae None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:22.476591 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:22.483041 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce4d17e-b511-498f-9fcd-17af32ae6fae None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:22.519019 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d89eedf-1af2-452b-8362-732115c1de6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:22.519697 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d89eedf-1af2-452b-8362-732115c1de6f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:22.520032 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d89eedf-1af2-452b-8362-732115c1de6f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:22.520543 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d89eedf-1af2-452b-8362-732115c1de6f None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:22.521657 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d89eedf-1af2-452b-8362-732115c1de6f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:22.522149 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d89eedf-1af2-452b-8362-732115c1de6f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:22.523563 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d89eedf-1af2-452b-8362-732115c1de6f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:22.537209 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 370/750] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Jun 10 12:45:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:22.543227 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee9c9d3-80ab-4f28-be40-3e9dfcdbba13 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:22.575021 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:22.654822 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee9c9d3-80ab-4f28-be40-3e9dfcdbba13 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:22.703384 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f048521-57a7-4634-8c54-b73dbe88441f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:22.704241 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f048521-57a7-4634-8c54-b73dbe88441f None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:22.704522 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f048521-57a7-4634-8c54-b73dbe88441f None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:22.704797 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f048521-57a7-4634-8c54-b73dbe88441f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:22.711890 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f048521-57a7-4634-8c54-b73dbe88441f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:22.713346 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f048521-57a7-4634-8c54-b73dbe88441f None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:22.731593 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-0f048521-57a7-4634-8c54-b73dbe88441f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 10 12:45:22.731593 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:22.731593 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:22.731593 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:22.731593 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:22.731593 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:22.731593 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:22.731593 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:22.731593 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:22.731593 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:22.731593 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:22.731593 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:22.731593 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 10 12:45:22.731593 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 10 12:45:22.731593 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 10 12:45:22.732573 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 10 12:45:22.732573 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:22.732573 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:22.732573 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:22.732573 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:22.732573 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:22.732573 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:22.732573 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:22.732573 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:22.732573 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:22.732573 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:22.732573 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:22.732573 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:22.732573 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:22.732573 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:22.732573 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:22.732573 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:22.733531 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 10 12:45:22.733531 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 10 12:45:22.733531 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 10 12:45:22.733531 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 10 12:45:22.733531 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 10 12:45:22.733531 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 10 12:45:22.733531 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 10 12:45:22.733531 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:22.733531 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 381/751] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Jun 10 12:45:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:22.740257 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d89eedf-1af2-452b-8362-732115c1de6f None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:22.905112 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d89eedf-1af2-452b-8362-732115c1de6f None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:22.957517 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5202a9c8-04cd-488a-9044-85519f779c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:22.958196 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5202a9c8-04cd-488a-9044-85519f779c8f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:22.958485 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5202a9c8-04cd-488a-9044-85519f779c8f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:22.958766 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5202a9c8-04cd-488a-9044-85519f779c8f None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:22.959563 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5202a9c8-04cd-488a-9044-85519f779c8f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:22.959889 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5202a9c8-04cd-488a-9044-85519f779c8f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:22.961872 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5202a9c8-04cd-488a-9044-85519f779c8f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:22.983470 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 371/752] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Jun 10 12:45:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:22.989892 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f048521-57a7-4634-8c54-b73dbe88441f None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:23.061165 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe547c3-a589-48a7-a387-52fd7a6fb32d None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:23.061481 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe547c3-a589-48a7-a387-52fd7a6fb32d None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:23.061736 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe547c3-a589-48a7-a387-52fd7a6fb32d None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:23.063400 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 372/753] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:45:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:45:23.068675 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c394a1f-d99b-4983-8b19-d70bcf0e570a None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:23.068972 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c394a1f-d99b-4983-8b19-d70bcf0e570a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:23.069261 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c394a1f-d99b-4983-8b19-d70bcf0e570a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:23.095116 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-2c394a1f-d99b-4983-8b19-d70bcf0e570a None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:45:23.097419 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2c394a1f-d99b-4983-8b19-d70bcf0e570a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:45:23.126209 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:23.160254 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:23.183989 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f048521-57a7-4634-8c54-b73dbe88441f None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:23.223106 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 373/754] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:45:23 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 10 12:45:23.258584 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c394a1f-d99b-4983-8b19-d70bcf0e570a None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:23.260425 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c90d0d3-4ba2-423a-83db-d422e803f548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:23.261099 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c90d0d3-4ba2-423a-83db-d422e803f548 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:23.261389 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c90d0d3-4ba2-423a-83db-d422e803f548 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:23.261673 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c90d0d3-4ba2-423a-83db-d422e803f548 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:23.262524 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c90d0d3-4ba2-423a-83db-d422e803f548 None admin] RBAC: Adding query filter params (role.id=b43d5a8247e244869c2c70f986f42e8a, scope.project.id=73ea5b23a2d34544ab493f4c5e4034b8, user.id=fb63b9cd74bf438aa25b72cef8337fec) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:23.262842 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c90d0d3-4ba2-423a-83db-d422e803f548 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:23.264315 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c90d0d3-4ba2-423a-83db-d422e803f548 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:23.278025 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 382/755] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Jun 10 12:45:22 2022] GET /identity/v3/role_assignments?user.id=fb63b9cd74bf438aa25b72cef8337fec&scope.project.id=73ea5b23a2d34544ab493f4c5e4034b8&role.id=b43d5a8247e244869c2c70f986f42e8a => generated 259 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:23.424369 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c394a1f-d99b-4983-8b19-d70bcf0e570a None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:23.469805 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a44c442-307c-4cf5-95c3-72bcd8ee0f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:23.470636 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a44c442-307c-4cf5-95c3-72bcd8ee0f83 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:23.470925 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a44c442-307c-4cf5-95c3-72bcd8ee0f83 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:23.471207 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a44c442-307c-4cf5-95c3-72bcd8ee0f83 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:23.485041 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a44c442-307c-4cf5-95c3-72bcd8ee0f83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:45:23.493169 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-3a44c442-307c-4cf5-95c3-72bcd8ee0f83 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 10 12:45:23.493169 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:23.493169 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:23.493169 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:23.493169 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:23.493169 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:23.493169 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:23.493169 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:23.493169 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:23.493169 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:23.493169 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:23.493169 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:23.493169 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:45:23.493169 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:45:23.493169 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:45:23.493169 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:45:23.494177 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:45:23.494177 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:45:23.494177 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:45:23.494177 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:45:23.494177 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:23.494177 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:23.494177 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:23.494177 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:23.494177 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:23.494177 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:23.494177 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:23.494177 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:23.494177 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:23.494177 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:23.494177 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:23.494177 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:23.495034 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:23.495034 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:23.495034 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:23.495034 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:23.495034 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:45:23.495034 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:45:23.495034 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:45:23.495034 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:45:23.495034 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:45:23.495034 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:45:23.495034 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 10 12:45:23.495034 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:23.495034 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 374/756] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Jun 10 12:45:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:23.500613 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c90d0d3-4ba2-423a-83db-d422e803f548 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:23.552037 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47560ea7-a7e8-4ff0-bba1-34f37c4a49e5 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:23.552037 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47560ea7-a7e8-4ff0-bba1-34f37c4a49e5 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:23.552037 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47560ea7-a7e8-4ff0-bba1-34f37c4a49e5 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:23.552037 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 375/757] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:45:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 10 12:45:23.553000 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c679ca-fd54-463c-8ef5-dac6eeebd553 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:23.553294 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c679ca-fd54-463c-8ef5-dac6eeebd553 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:23.553578 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c679ca-fd54-463c-8ef5-dac6eeebd553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:23.565192 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:23.571770 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-84c679ca-fd54-463c-8ef5-dac6eeebd553 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:45:23.573261 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-84c679ca-fd54-463c-8ef5-dac6eeebd553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:45:23.639216 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c90d0d3-4ba2-423a-83db-d422e803f548 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:23.677873 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04661b4-4607-4d67-9ea0-e3f30ebdfe93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:23.679602 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04661b4-4607-4d67-9ea0-e3f30ebdfe93 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:23.680120 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04661b4-4607-4d67-9ea0-e3f30ebdfe93 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:23.680626 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04661b4-4607-4d67-9ea0-e3f30ebdfe93 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:23.681567 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04661b4-4607-4d67-9ea0-e3f30ebdfe93 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:23.682073 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04661b4-4607-4d67-9ea0-e3f30ebdfe93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:23.683783 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04661b4-4607-4d67-9ea0-e3f30ebdfe93 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:23.695287 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 383/758] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Jun 10 12:45:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:23.700661 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04661b4-4607-4d67-9ea0-e3f30ebdfe93 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:23.714368 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:23.734660 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 376/759] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:45:23 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 10 12:45:23.780685 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c679ca-fd54-463c-8ef5-dac6eeebd553 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:23.868993 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04661b4-4607-4d67-9ea0-e3f30ebdfe93 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:23.923191 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c679ca-fd54-463c-8ef5-dac6eeebd553 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:23.943072 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf95f2f6-12f1-4633-b4ab-9fd445fbd4ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:23.943891 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf95f2f6-12f1-4633-b4ab-9fd445fbd4ec None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:23.944180 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf95f2f6-12f1-4633-b4ab-9fd445fbd4ec None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:23.945011 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf95f2f6-12f1-4633-b4ab-9fd445fbd4ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:23.969938 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf95f2f6-12f1-4633-b4ab-9fd445fbd4ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:23.971396 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf95f2f6-12f1-4633-b4ab-9fd445fbd4ec None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:23.984297 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f31478-5cf8-41ce-afc0-3770ce9b320e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:23.985122 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f31478-5cf8-41ce-afc0-3770ce9b320e None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:23.985409 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f31478-5cf8-41ce-afc0-3770ce9b320e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:23.985690 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f31478-5cf8-41ce-afc0-3770ce9b320e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:23.986348 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f31478-5cf8-41ce-afc0-3770ce9b320e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:23.987708 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f31478-5cf8-41ce-afc0-3770ce9b320e None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:23.996698 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:24.009183 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-cf95f2f6-12f1-4633-b4ab-9fd445fbd4ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:24.009183 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:24.009183 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:24.009183 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:24.009183 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:24.009183 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:24.009183 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:24.009183 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:24.009183 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:24.009183 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:24.009183 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:24.009183 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:24.009183 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 10 12:45:24.009183 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 10 12:45:24.009183 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 10 12:45:24.012661 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 10 12:45:24.012661 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:24.012661 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:24.012661 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:24.012661 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:24.012661 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:24.012661 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:24.012661 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:24.012661 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:24.012661 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:24.012661 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:24.012661 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:24.012661 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:24.012661 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:24.012661 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:24.012661 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:24.012661 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:24.013570 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 10 12:45:24.013570 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 10 12:45:24.013570 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:24.013570 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:24.013570 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 384/760] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:45:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 10 12:45:24.013855 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-84f31478-5cf8-41ce-afc0-3770ce9b320e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 10 12:45:24.013855 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:24.013855 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 10 12:45:24.013855 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 10 12:45:24.013855 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 10 12:45:24.013855 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 10 12:45:24.013855 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 10 12:45:24.013855 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 10 12:45:24.013855 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 10 12:45:24.013855 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:24.013855 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 10 12:45:24.013855 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:24.013855 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:24.013855 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:24.013855 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:24.013855 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:24.014720 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:24.014720 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:24.014720 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:24.014720 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:24.014720 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:24.014720 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:24.014720 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:24.014720 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 10 12:45:24.014720 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 10 12:45:24.014720 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 10 12:45:24.014720 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 10 12:45:24.014720 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:24.014720 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:24.014720 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:24.014720 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:24.014720 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:24.015591 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:24.015591 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:24.015591 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:24.015591 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:24.015591 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:24.015591 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:24.015591 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:24.015591 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:24.015591 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:24.015591 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:24.015591 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:24.015591 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 10 12:45:24.015591 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 10 12:45:24.015591 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 10 12:45:24.015591 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:24.015591 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 377/761] 10.222.0.27 () {56 vars in 1129 bytes} [Fri Jun 10 12:45:23 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 10 12:45:24.019234 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf95f2f6-12f1-4633-b4ab-9fd445fbd4ec None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:24.021415 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f31478-5cf8-41ce-afc0-3770ce9b320e None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:24.146271 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:24.206145 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f31478-5cf8-41ce-afc0-3770ce9b320e None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:24.217347 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf95f2f6-12f1-4633-b4ab-9fd445fbd4ec None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:24.256691 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa972d0-19cc-4a6a-9285-0d3e506eba6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:24.257404 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa972d0-19cc-4a6a-9285-0d3e506eba6d None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:24.257689 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa972d0-19cc-4a6a-9285-0d3e506eba6d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:24.257968 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa972d0-19cc-4a6a-9285-0d3e506eba6d None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:24.259890 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa972d0-19cc-4a6a-9285-0d3e506eba6d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:24.259890 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa972d0-19cc-4a6a-9285-0d3e506eba6d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:24.260376 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa972d0-19cc-4a6a-9285-0d3e506eba6d None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:24.265469 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 378/762] 10.222.0.27 () {56 vars in 1124 bytes} [Fri Jun 10 12:45:24 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:24.272342 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa972d0-19cc-4a6a-9285-0d3e506eba6d None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:24.291440 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77379fc-4939-4b9a-9924-67d5c6e5929b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:24.292247 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77379fc-4939-4b9a-9924-67d5c6e5929b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:24.292596 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77379fc-4939-4b9a-9924-67d5c6e5929b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:24.293163 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77379fc-4939-4b9a-9924-67d5c6e5929b None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:24.293905 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77379fc-4939-4b9a-9924-67d5c6e5929b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:24.294213 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77379fc-4939-4b9a-9924-67d5c6e5929b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:24.295592 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77379fc-4939-4b9a-9924-67d5c6e5929b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:24.309479 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 385/763] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Jun 10 12:45:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 12:45:24.315589 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77379fc-4939-4b9a-9924-67d5c6e5929b None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:24.411663 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:24.470643 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa972d0-19cc-4a6a-9285-0d3e506eba6d None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:24.493605 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77379fc-4939-4b9a-9924-67d5c6e5929b None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:24.519732 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:24.537516 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae14d4ec-13bf-4ca2-b488-b9113b2375a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:24.538829 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae14d4ec-13bf-4ca2-b488-b9113b2375a1 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:24.539116 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae14d4ec-13bf-4ca2-b488-b9113b2375a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:24.539394 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae14d4ec-13bf-4ca2-b488-b9113b2375a1 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:24.540901 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae14d4ec-13bf-4ca2-b488-b9113b2375a1 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:24.541222 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae14d4ec-13bf-4ca2-b488-b9113b2375a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:24.543243 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba28f2a5-8b90-43d0-8427-32a236d5b5c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:24.544111 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba28f2a5-8b90-43d0-8427-32a236d5b5c0 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:24.544391 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba28f2a5-8b90-43d0-8427-32a236d5b5c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:24.544666 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba28f2a5-8b90-43d0-8427-32a236d5b5c0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:24.545607 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae14d4ec-13bf-4ca2-b488-b9113b2375a1 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:24.558767 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 379/764] 10.222.0.27 () {56 vars in 1124 bytes} [Fri Jun 10 12:45:24 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:24.559992 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba28f2a5-8b90-43d0-8427-32a236d5b5c0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:24.561486 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba28f2a5-8b90-43d0-8427-32a236d5b5c0 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:24.564948 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae14d4ec-13bf-4ca2-b488-b9113b2375a1 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:24.588087 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-ba28f2a5-8b90-43d0-8427-32a236d5b5c0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 10 12:45:24.588087 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:24.588087 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:24.588087 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:24.588087 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:24.588087 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:24.588087 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:24.588087 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:24.588087 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:24.588087 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:24.588087 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:24.588087 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:24.588087 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 10 12:45:24.588087 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 10 12:45:24.588087 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 10 12:45:24.588087 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 10 12:45:24.589120 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:24.589120 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:24.589120 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 10 12:45:24.589120 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 10 12:45:24.589120 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 10 12:45:24.589120 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 10 12:45:24.589120 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:24.589120 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:24.589120 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:24.589120 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:24.589120 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:24.589120 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:24.589120 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:24.589120 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:24.589120 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:24.589120 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:24.589120 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:24.589120 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:24.590702 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:24.590702 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:24.590702 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 10 12:45:24.590702 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 10 12:45:24.590702 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 10 12:45:24.590702 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 10 12:45:24.590702 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 10 12:45:24.590702 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 10 12:45:24.590702 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 10 12:45:24.590702 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:24.590702 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 386/765] 10.222.0.27 () {56 vars in 1093 bytes} [Fri Jun 10 12:45:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 10 12:45:24.599897 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba28f2a5-8b90-43d0-8427-32a236d5b5c0 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:24.716687 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae14d4ec-13bf-4ca2-b488-b9113b2375a1 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:24.734767 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba28f2a5-8b90-43d0-8427-32a236d5b5c0 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:24.749290 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:24.757898 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d80932-2a4c-4f70-9811-76aae2213300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:24.758679 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d80932-2a4c-4f70-9811-76aae2213300 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:24.764109 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d80932-2a4c-4f70-9811-76aae2213300 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:24.764109 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d80932-2a4c-4f70-9811-76aae2213300 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:24.764109 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d80932-2a4c-4f70-9811-76aae2213300 None admin] RBAC: Adding query filter params (interface=public, service_id=f10f67aa642c497aa4434f615bbae613, region_id=RegionOne) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:24.764109 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d80932-2a4c-4f70-9811-76aae2213300 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:24.764109 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d80932-2a4c-4f70-9811-76aae2213300 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:24.775211 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 380/766] 10.222.0.27 () {56 vars in 1251 bytes} [Fri Jun 10 12:45:24 2022] GET /identity/v3/endpoints?service_id=f10f67aa642c497aa4434f615bbae613&interface=public®ion_id=RegionOne => generated 194 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 12:45:24.777440 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4265d99-6d81-4a9d-acf6-2461629fa26b None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:24.777988 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4265d99-6d81-4a9d-acf6-2461629fa26b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:24.778384 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4e17fa-bd41-4548-b971-cb416e4922fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:24.778754 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4265d99-6d81-4a9d-acf6-2461629fa26b None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:24.779092 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4e17fa-bd41-4548-b971-cb416e4922fb None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:24.779374 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4e17fa-bd41-4548-b971-cb416e4922fb None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:24.779650 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4e17fa-bd41-4548-b971-cb416e4922fb None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:24.780565 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4e17fa-bd41-4548-b971-cb416e4922fb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:24.780879 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4e17fa-bd41-4548-b971-cb416e4922fb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:24.785911 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4e17fa-bd41-4548-b971-cb416e4922fb None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:24.786148 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 381/767] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:45:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:45:24.786148 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4265d99-6d81-4a9d-acf6-2461629fa26b None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:24.811963 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 387/768] 10.222.0.27 () {56 vars in 1133 bytes} [Fri Jun 10 12:45:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:24.813848 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd008313-2445-4d89-b6f6-414992f0c188 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:24.814462 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd008313-2445-4d89-b6f6-414992f0c188 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:24.816156 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd008313-2445-4d89-b6f6-414992f0c188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:24.838297 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-cd008313-2445-4d89-b6f6-414992f0c188 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:45:24.839797 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-cd008313-2445-4d89-b6f6-414992f0c188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:45:24.878439 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:24.911619 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4265d99-6d81-4a9d-acf6-2461629fa26b None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:24.920147 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 388/769] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:45:24 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 10 12:45:24.920147 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd008313-2445-4d89-b6f6-414992f0c188 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:24.954197 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddb7c9f-35a1-4853-afd7-3f4e7ef1ad97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:24.954864 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddb7c9f-35a1-4853-afd7-3f4e7ef1ad97 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:24.955150 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddb7c9f-35a1-4853-afd7-3f4e7ef1ad97 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:24.955429 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddb7c9f-35a1-4853-afd7-3f4e7ef1ad97 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:24.956164 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddb7c9f-35a1-4853-afd7-3f4e7ef1ad97 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:24.957539 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddb7c9f-35a1-4853-afd7-3f4e7ef1ad97 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:24.963332 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 382/770] 10.222.0.27 () {56 vars in 1087 bytes} [Fri Jun 10 12:45:24 2022] GET /identity/v3/services => generated 772 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:24.970426 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1192c30c-f1ac-47b9-ad1f-ee61a2b0ed20 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:24.971368 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1192c30c-f1ac-47b9-ad1f-ee61a2b0ed20 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:24.971898 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1192c30c-f1ac-47b9-ad1f-ee61a2b0ed20 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:24.979512 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 383/771] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:45:24 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:45:24.981125 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1192c30c-f1ac-47b9-ad1f-ee61a2b0ed20 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:25.053802 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:25.059557 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd008313-2445-4d89-b6f6-414992f0c188 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:25.107926 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f90662-7616-41eb-9c61-4c012cf61826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:25.108851 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f90662-7616-41eb-9c61-4c012cf61826 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:25.109143 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f90662-7616-41eb-9c61-4c012cf61826 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:25.109424 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f90662-7616-41eb-9c61-4c012cf61826 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:25.114913 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1192c30c-f1ac-47b9-ad1f-ee61a2b0ed20 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:25.122155 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f90662-7616-41eb-9c61-4c012cf61826 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:25.123626 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f90662-7616-41eb-9c61-4c012cf61826 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:25.146402 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-83f90662-7616-41eb-9c61-4c012cf61826 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:25.146402 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:25.146402 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:25.146402 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:25.146402 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:25.146402 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:25.146402 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:25.146402 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:25.146402 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:25.146402 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:25.146402 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:25.146402 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:25.146402 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 10 12:45:25.146402 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 10 12:45:25.146402 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 10 12:45:25.147400 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 10 12:45:25.147400 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:25.147400 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:25.147400 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:25.147400 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:25.147400 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:25.147400 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:25.147400 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:25.147400 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:25.147400 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:25.147400 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:25.147400 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:25.147400 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:25.147400 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:25.147400 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:25.147400 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:25.147400 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:25.148400 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 10 12:45:25.148400 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 10 12:45:25.148400 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:25.148400 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:25.148400 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 389/772] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:45:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:25.150402 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7d73ff-eb9e-4f6f-920e-d70be0b892ef None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:25.150402 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7d73ff-eb9e-4f6f-920e-d70be0b892ef None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:25.150402 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7d73ff-eb9e-4f6f-920e-d70be0b892ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:25.164702 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0074c4-ba70-42ac-afde-9e11eaa74296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:25.167676 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0074c4-ba70-42ac-afde-9e11eaa74296 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:25.167676 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0074c4-ba70-42ac-afde-9e11eaa74296 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:25.167676 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0074c4-ba70-42ac-afde-9e11eaa74296 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:25.173370 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-1e7d73ff-eb9e-4f6f-920e-d70be0b892ef None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:45:25.174569 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-1e7d73ff-eb9e-4f6f-920e-d70be0b892ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:45:25.179931 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0074c4-ba70-42ac-afde-9e11eaa74296 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:45:25.187851 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-cb0074c4-ba70-42ac-afde-9e11eaa74296 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 10 12:45:25.187851 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:25.187851 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:25.187851 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:25.187851 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:25.187851 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:25.187851 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:25.187851 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:25.187851 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:25.187851 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:25.187851 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:25.187851 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:25.187851 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:45:25.187851 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:45:25.187851 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:45:25.187851 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:45:25.188903 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:45:25.188903 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:45:25.188903 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:45:25.188903 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:45:25.188903 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:25.188903 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:25.188903 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:25.188903 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:25.188903 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:25.188903 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:25.188903 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:25.188903 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:25.188903 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:25.188903 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:25.188903 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:25.188903 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:25.189765 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:25.189765 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:25.189765 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:25.189765 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:25.189765 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:45:25.189765 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:45:25.189765 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:45:25.189765 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:45:25.189765 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:45:25.189765 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:45:25.189765 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 10 12:45:25.189765 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:25.192545 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 384/773] 10.222.0.27 () {56 vars in 1102 bytes} [Fri Jun 10 12:45:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:25.193538 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0074c4-ba70-42ac-afde-9e11eaa74296 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:25.199367 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:25.271777 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 390/774] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:45:25 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 10 12:45:25.276996 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7d73ff-eb9e-4f6f-920e-d70be0b892ef None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:25.314617 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0074c4-ba70-42ac-afde-9e11eaa74296 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:25.333105 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:25.353031 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5f18fe-add3-47cc-b59f-503b274b7633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:25.353712 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5f18fe-add3-47cc-b59f-503b274b7633 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:25.354005 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5f18fe-add3-47cc-b59f-503b274b7633 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:25.354414 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5f18fe-add3-47cc-b59f-503b274b7633 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:25.355143 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5f18fe-add3-47cc-b59f-503b274b7633 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:25.355452 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5f18fe-add3-47cc-b59f-503b274b7633 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:25.357006 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5f18fe-add3-47cc-b59f-503b274b7633 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:25.372095 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 385/775] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Jun 10 12:45:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:25.373026 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5f18fe-add3-47cc-b59f-503b274b7633 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:25.401109 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7d73ff-eb9e-4f6f-920e-d70be0b892ef None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:25.438044 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4521197-ed37-4ddb-a09b-6254e889816c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:25.439131 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4521197-ed37-4ddb-a09b-6254e889816c None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:25.439623 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4521197-ed37-4ddb-a09b-6254e889816c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:25.440148 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4521197-ed37-4ddb-a09b-6254e889816c None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:25.441228 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4521197-ed37-4ddb-a09b-6254e889816c None admin] RBAC: Adding query filter params (name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:25.441776 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4521197-ed37-4ddb-a09b-6254e889816c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:25.443439 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4521197-ed37-4ddb-a09b-6254e889816c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:25.457806 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 391/776] 10.222.0.27 () {56 vars in 1103 bytes} [Fri Jun 10 12:45:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:25.459031 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4521197-ed37-4ddb-a09b-6254e889816c None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:25.476624 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:25.481547 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5f18fe-add3-47cc-b59f-503b274b7633 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:25.518260 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0701358f-ee1c-4911-a684-120ace44f972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:25.519074 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0701358f-ee1c-4911-a684-120ace44f972 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:25.519364 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0701358f-ee1c-4911-a684-120ace44f972 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:25.519647 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0701358f-ee1c-4911-a684-120ace44f972 None admin] PATH_INFO: `/v3/services/network` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:25.520468 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0701358f-ee1c-4911-a684-120ace44f972 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:25.521991 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0701358f-ee1c-4911-a684-120ace44f972 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:25.534508 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-0701358f-ee1c-4911-a684-120ace44f972 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 10 12:45:25.534508 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:25.534508 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 10 12:45:25.534508 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 10 12:45:25.534508 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 10 12:45:25.534508 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 10 12:45:25.534508 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 10 12:45:25.534508 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 10 12:45:25.534508 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 10 12:45:25.534508 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:25.534508 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 10 12:45:25.534508 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:25.534508 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:25.534508 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:25.534508 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:25.534508 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:25.534508 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:25.535544 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:25.535544 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:25.535544 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:25.535544 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:25.535544 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:25.535544 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:25.535544 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 10 12:45:25.535544 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 10 12:45:25.535544 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 10 12:45:25.535544 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 10 12:45:25.535544 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:25.535544 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:25.535544 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:25.535544 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:25.535544 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:25.535544 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:25.540916 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:25.540916 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:25.540916 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:25.540916 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:25.540916 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:25.540916 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:25.540916 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:25.540916 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:25.540916 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:25.540916 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:25.540916 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 10 12:45:25.540916 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 10 12:45:25.540916 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 10 12:45:25.540916 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:25.540916 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 386/777] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Jun 10 12:45:25 2022] GET /identity/v3/services/network => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:25.541761 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0701358f-ee1c-4911-a684-120ace44f972 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:25.578768 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4521197-ed37-4ddb-a09b-6254e889816c None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:25.617842 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacaf113-3d48-447c-84fc-08677bf66acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:25.619029 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacaf113-3d48-447c-84fc-08677bf66acf None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:25.619565 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacaf113-3d48-447c-84fc-08677bf66acf None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:25.620112 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacaf113-3d48-447c-84fc-08677bf66acf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:25.627454 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacaf113-3d48-447c-84fc-08677bf66acf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:25.629243 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacaf113-3d48-447c-84fc-08677bf66acf None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:25.637010 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:25.643852 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-cacaf113-3d48-447c-84fc-08677bf66acf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 10 12:45:25.643852 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:25.643852 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:25.643852 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:25.643852 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:25.643852 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:25.643852 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:25.643852 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:25.643852 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:25.643852 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:25.643852 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:25.643852 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:25.643852 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 10 12:45:25.643852 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 10 12:45:25.643852 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 10 12:45:25.645005 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 10 12:45:25.645005 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:25.645005 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:25.645005 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:25.645005 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:25.645005 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:25.645005 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:25.645005 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:25.645005 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:25.645005 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:25.645005 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:25.645005 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:25.645005 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:25.645005 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:25.645005 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:25.645005 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:25.645005 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:25.646069 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 10 12:45:25.646069 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 10 12:45:25.646069 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 10 12:45:25.646069 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 10 12:45:25.646069 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 10 12:45:25.646069 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 10 12:45:25.646069 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 10 12:45:25.646069 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:25.646069 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 392/778] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Jun 10 12:45:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:25.646719 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacaf113-3d48-447c-84fc-08677bf66acf None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:25.652213 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0701358f-ee1c-4911-a684-120ace44f972 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:25.691442 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424d4cd5-6343-4641-9234-9f0e46daee57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:25.692264 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424d4cd5-6343-4641-9234-9f0e46daee57 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:25.692546 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424d4cd5-6343-4641-9234-9f0e46daee57 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:25.693222 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424d4cd5-6343-4641-9234-9f0e46daee57 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:25.704982 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424d4cd5-6343-4641-9234-9f0e46daee57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:25.706428 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424d4cd5-6343-4641-9234-9f0e46daee57 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:25.728768 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-424d4cd5-6343-4641-9234-9f0e46daee57 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:25.728768 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:25.728768 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:25.728768 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:25.728768 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:25.728768 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:25.728768 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:25.728768 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:25.728768 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:25.728768 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:25.728768 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:25.728768 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:25.728768 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 10 12:45:25.728768 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 10 12:45:25.728768 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 10 12:45:25.729703 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 10 12:45:25.729703 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:25.729703 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:25.729703 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:25.729703 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:25.729703 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:25.729703 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:25.729703 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:25.729703 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:25.729703 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:25.729703 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:25.729703 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:25.729703 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:25.729703 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:25.729703 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:25.729703 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:25.729703 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:25.730590 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 10 12:45:25.730590 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 10 12:45:25.730590 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:25.730590 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:25.730590 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 387/779] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:45:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:25.731183 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec6d872-9949-47d8-96cc-eb7a17ba1e98 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:25.731469 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec6d872-9949-47d8-96cc-eb7a17ba1e98 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:25.731752 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec6d872-9949-47d8-96cc-eb7a17ba1e98 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:25.733342 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 388/780] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:45:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:45:25.733941 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec6d872-9949-47d8-96cc-eb7a17ba1e98 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:25.736679 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:25.763091 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacaf113-3d48-447c-84fc-08677bf66acf None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:25.797497 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ea5984-f498-436e-b5a2-1f4b3ee14df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:25.798058 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ea5984-f498-436e-b5a2-1f4b3ee14df7 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:25.798342 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ea5984-f498-436e-b5a2-1f4b3ee14df7 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:25.798621 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ea5984-f498-436e-b5a2-1f4b3ee14df7 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:25.799326 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ea5984-f498-436e-b5a2-1f4b3ee14df7 None admin] RBAC: Adding query filter params (name=network) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:25.799638 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ea5984-f498-436e-b5a2-1f4b3ee14df7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:25.801038 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ea5984-f498-436e-b5a2-1f4b3ee14df7 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:25.810541 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 393/781] 10.222.0.27 () {56 vars in 1112 bytes} [Fri Jun 10 12:45:25 2022] GET /identity/v3/services?name=network => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:25.810541 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ea5984-f498-436e-b5a2-1f4b3ee14df7 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:25.839539 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:25.849797 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec6d872-9949-47d8-96cc-eb7a17ba1e98 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:25.891150 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bf6ad4-e813-4ef4-9a63-68fd562d314c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:25.891908 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bf6ad4-e813-4ef4-9a63-68fd562d314c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:25.892260 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bf6ad4-e813-4ef4-9a63-68fd562d314c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:25.892600 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bf6ad4-e813-4ef4-9a63-68fd562d314c None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:25.893451 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bf6ad4-e813-4ef4-9a63-68fd562d314c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:25.893765 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bf6ad4-e813-4ef4-9a63-68fd562d314c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:25.895182 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bf6ad4-e813-4ef4-9a63-68fd562d314c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:25.907232 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 389/782] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Jun 10 12:45:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:25.908319 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbdc6ef-0b53-44d5-8509-14c760aa02da None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:25.908606 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbdc6ef-0b53-44d5-8509-14c760aa02da None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:25.908884 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbdc6ef-0b53-44d5-8509-14c760aa02da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:25.924371 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ea5984-f498-436e-b5a2-1f4b3ee14df7 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:25.925477 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-2cbdc6ef-0b53-44d5-8509-14c760aa02da None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:45:25.926666 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2cbdc6ef-0b53-44d5-8509-14c760aa02da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:45:25.961931 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c806d874-edc6-401e-ae13-1fb89008d104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:25.962678 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c806d874-edc6-401e-ae13-1fb89008d104 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:25.963030 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c806d874-edc6-401e-ae13-1fb89008d104 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:25.963382 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c806d874-edc6-401e-ae13-1fb89008d104 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:25.964201 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c806d874-edc6-401e-ae13-1fb89008d104 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:25.964603 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c806d874-edc6-401e-ae13-1fb89008d104 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:25.965985 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c806d874-edc6-401e-ae13-1fb89008d104 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:25.973668 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:25.979540 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 394/783] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Jun 10 12:45:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:25.980429 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c806d874-edc6-401e-ae13-1fb89008d104 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:26.004728 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 390/784] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:45:25 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 10 12:45:26.016081 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbdc6ef-0b53-44d5-8509-14c760aa02da None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:26.097360 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c806d874-edc6-401e-ae13-1fb89008d104 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:26.098356 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:26.119168 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbdc6ef-0b53-44d5-8509-14c760aa02da None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:26.133862 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ecae29-42eb-44a3-8bf6-cfc7c88e17f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:26.134613 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ecae29-42eb-44a3-8bf6-cfc7c88e17f5 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:26.134972 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ecae29-42eb-44a3-8bf6-cfc7c88e17f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:26.135312 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ecae29-42eb-44a3-8bf6-cfc7c88e17f5 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:26.136120 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ecae29-42eb-44a3-8bf6-cfc7c88e17f5 None admin] RBAC: Adding query filter params (type=network) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:26.136491 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ecae29-42eb-44a3-8bf6-cfc7c88e17f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:26.137922 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ecae29-42eb-44a3-8bf6-cfc7c88e17f5 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:26.143961 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 395/785] 10.222.0.27 () {56 vars in 1112 bytes} [Fri Jun 10 12:45:25 2022] GET /identity/v3/services?type=network => generated 350 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:26.144775 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ecae29-42eb-44a3-8bf6-cfc7c88e17f5 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:26.162781 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4e06ee-50b6-40e1-8c19-c0a50e451dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:26.164393 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4e06ee-50b6-40e1-8c19-c0a50e451dc2 None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:26.164677 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4e06ee-50b6-40e1-8c19-c0a50e451dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:26.164954 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4e06ee-50b6-40e1-8c19-c0a50e451dc2 None admin] PATH_INFO: `/v3/projects/73ea5b23a2d34544ab493f4c5e4034b8/users/8bd6caa461e84100a3d5fe5af5d80f3e/roles/b43d5a8247e244869c2c70f986f42e8a` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:26.167065 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4e06ee-50b6-40e1-8c19-c0a50e451dc2 None admin] RBAC: Authorizing `identity:create_grant(project_id=73ea5b23a2d34544ab493f4c5e4034b8, user_id=8bd6caa461e84100a3d5fe5af5d80f3e, role_id=b43d5a8247e244869c2c70f986f42e8a)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:26.168660 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4e06ee-50b6-40e1-8c19-c0a50e451dc2 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:26.206325 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:26.243527 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ecae29-42eb-44a3-8bf6-cfc7c88e17f5 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:26.251908 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 391/786] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Jun 10 12:45:26 2022] PUT /identity/v3/projects/73ea5b23a2d34544ab493f4c5e4034b8/users/8bd6caa461e84100a3d5fe5af5d80f3e/roles/b43d5a8247e244869c2c70f986f42e8a => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 12:45:26.251908 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4e06ee-50b6-40e1-8c19-c0a50e451dc2 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:26.287642 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d677926d-0886-4c3c-aee2-ace8fbb577e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:26.288622 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d677926d-0886-4c3c-aee2-ace8fbb577e0 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:26.288955 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d677926d-0886-4c3c-aee2-ace8fbb577e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:26.289279 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d677926d-0886-4c3c-aee2-ace8fbb577e0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:26.298571 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d677926d-0886-4c3c-aee2-ace8fbb577e0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:26.300098 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d677926d-0886-4c3c-aee2-ace8fbb577e0 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:26.316773 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-d677926d-0886-4c3c-aee2-ace8fbb577e0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 10 12:45:26.316773 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:26.316773 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:26.316773 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:26.316773 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:26.316773 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:26.316773 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:26.316773 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:26.316773 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:26.316773 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:26.316773 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:26.316773 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:26.316773 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 10 12:45:26.316773 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 10 12:45:26.316773 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 10 12:45:26.316773 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 10 12:45:26.318024 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:26.318024 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:26.318024 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 10 12:45:26.318024 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 10 12:45:26.318024 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 10 12:45:26.318024 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 10 12:45:26.318024 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:26.318024 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:26.318024 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:26.318024 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:26.318024 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:26.318024 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:26.318024 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:26.318024 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:26.318024 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:26.318024 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:26.318024 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:26.318024 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:26.319828 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:26.319828 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:26.319828 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 10 12:45:26.319828 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 10 12:45:26.319828 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 10 12:45:26.319828 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 10 12:45:26.319828 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 10 12:45:26.319828 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 10 12:45:26.319828 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 10 12:45:26.319828 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:26.319828 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 396/787] 10.222.0.27 () {56 vars in 1117 bytes} [Fri Jun 10 12:45:26 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:26.319828 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d677926d-0886-4c3c-aee2-ace8fbb577e0 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:26.389556 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4e06ee-50b6-40e1-8c19-c0a50e451dc2 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:26.442476 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d677926d-0886-4c3c-aee2-ace8fbb577e0 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:26.447137 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa327ec7-637e-49f8-a224-7aeb4d0bd6c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:26.448111 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa327ec7-637e-49f8-a224-7aeb4d0bd6c2 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:26.448470 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa327ec7-637e-49f8-a224-7aeb4d0bd6c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:26.448819 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa327ec7-637e-49f8-a224-7aeb4d0bd6c2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:26.461645 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa327ec7-637e-49f8-a224-7aeb4d0bd6c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:45:26.469644 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:26.470690 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-fa327ec7-637e-49f8-a224-7aeb4d0bd6c2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 10 12:45:26.470690 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:26.470690 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:26.470690 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:26.470690 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:26.470690 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:26.470690 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:26.470690 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:26.470690 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:26.470690 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:26.470690 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:26.470690 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:26.470690 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:45:26.470690 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:45:26.470690 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:45:26.470690 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:45:26.471805 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:45:26.471805 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:45:26.471805 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:45:26.471805 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:45:26.471805 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:26.471805 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:26.471805 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:26.471805 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:26.471805 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:26.471805 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:26.471805 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:26.471805 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:26.471805 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:26.471805 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:26.471805 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:26.471805 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:26.473116 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:26.473116 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:26.473116 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:26.473116 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:26.473116 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:45:26.473116 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:45:26.473116 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:45:26.473116 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:45:26.473116 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:45:26.473116 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:45:26.473116 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 10 12:45:26.473116 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:26.476036 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 392/788] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Jun 10 12:45:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:26.477025 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa327ec7-637e-49f8-a224-7aeb4d0bd6c2 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:26.481023 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:26.483427 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea7a642-b988-4682-bc59-73864b88a088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:26.484131 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea7a642-b988-4682-bc59-73864b88a088 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:26.484416 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea7a642-b988-4682-bc59-73864b88a088 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:26.484694 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea7a642-b988-4682-bc59-73864b88a088 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:26.485497 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea7a642-b988-4682-bc59-73864b88a088 None admin] RBAC: Adding query filter params (interface=public, service_id=d79b4520598c484e86b343c7422bec60, region_id=RegionOne) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:26.486627 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea7a642-b988-4682-bc59-73864b88a088 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:26.488047 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea7a642-b988-4682-bc59-73864b88a088 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:26.496117 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 397/789] 10.222.0.27 () {56 vars in 1251 bytes} [Fri Jun 10 12:45:26 2022] GET /identity/v3/endpoints?service_id=d79b4520598c484e86b343c7422bec60&interface=public®ion_id=RegionOne => generated 194 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:26.497185 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea7a642-b988-4682-bc59-73864b88a088 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:26.603491 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea7a642-b988-4682-bc59-73864b88a088 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:26.614433 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa327ec7-637e-49f8-a224-7aeb4d0bd6c2 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:26.640777 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de56f9a-cd4e-471c-9fb4-148e07c42058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:26.641805 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de56f9a-cd4e-471c-9fb4-148e07c42058 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:26.642093 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de56f9a-cd4e-471c-9fb4-148e07c42058 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:26.642554 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de56f9a-cd4e-471c-9fb4-148e07c42058 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:26.643434 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de56f9a-cd4e-471c-9fb4-148e07c42058 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:26.643905 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de56f9a-cd4e-471c-9fb4-148e07c42058 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:26.645878 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de56f9a-cd4e-471c-9fb4-148e07c42058 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:26.656810 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 398/790] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Jun 10 12:45:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:26.657098 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29da4cc6-de46-4b73-ad48-49bf0ae751f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:26.657644 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de56f9a-cd4e-471c-9fb4-148e07c42058 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:26.657803 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29da4cc6-de46-4b73-ad48-49bf0ae751f7 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:26.658049 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29da4cc6-de46-4b73-ad48-49bf0ae751f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:26.658327 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29da4cc6-de46-4b73-ad48-49bf0ae751f7 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:26.659305 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29da4cc6-de46-4b73-ad48-49bf0ae751f7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:26.659620 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29da4cc6-de46-4b73-ad48-49bf0ae751f7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:26.662729 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29da4cc6-de46-4b73-ad48-49bf0ae751f7 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:26.686047 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 393/791] 10.222.0.27 () {56 vars in 1149 bytes} [Fri Jun 10 12:45:26 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:26.687632 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29da4cc6-de46-4b73-ad48-49bf0ae751f7 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:26.716830 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:26.759556 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:26.773928 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de56f9a-cd4e-471c-9fb4-148e07c42058 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:26.804868 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29da4cc6-de46-4b73-ad48-49bf0ae751f7 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:26.811808 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ca013c-acbd-4a95-a36c-06c0a9dd2a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:26.812734 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ca013c-acbd-4a95-a36c-06c0a9dd2a74 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:26.813199 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ca013c-acbd-4a95-a36c-06c0a9dd2a74 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:26.813776 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ca013c-acbd-4a95-a36c-06c0a9dd2a74 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:26.814667 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ca013c-acbd-4a95-a36c-06c0a9dd2a74 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:26.816467 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ca013c-acbd-4a95-a36c-06c0a9dd2a74 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:26.828243 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 399/792] 10.222.0.27 () {56 vars in 1087 bytes} [Fri Jun 10 12:45:26 2022] GET /identity/v3/services => generated 772 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:26.829307 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ca013c-acbd-4a95-a36c-06c0a9dd2a74 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:26.843338 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5451b0-5b5b-43a6-b9d3-d068b5f63428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:26.844485 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5451b0-5b5b-43a6-b9d3-d068b5f63428 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:26.845054 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5451b0-5b5b-43a6-b9d3-d068b5f63428 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:26.845565 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5451b0-5b5b-43a6-b9d3-d068b5f63428 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:26.857161 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5451b0-5b5b-43a6-b9d3-d068b5f63428 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:26.859300 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5451b0-5b5b-43a6-b9d3-d068b5f63428 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:26.881041 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-6a5451b0-5b5b-43a6-b9d3-d068b5f63428 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:26.881041 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:26.881041 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:26.881041 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:26.881041 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:26.881041 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:26.881041 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:26.881041 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:26.881041 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:26.881041 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:26.881041 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:26.881041 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:26.881041 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 10 12:45:26.881041 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 10 12:45:26.881041 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 10 12:45:26.882578 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 10 12:45:26.882578 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:26.882578 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:26.882578 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:26.882578 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:26.882578 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:26.882578 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:26.882578 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:26.882578 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:26.882578 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:26.882578 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:26.882578 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:26.882578 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:26.882578 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:26.882578 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:26.882578 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:26.882578 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:26.883485 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 10 12:45:26.883485 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 10 12:45:26.883485 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:26.883485 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:26.885862 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 394/793] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:45:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:26.892007 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5451b0-5b5b-43a6-b9d3-d068b5f63428 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:27.037854 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5451b0-5b5b-43a6-b9d3-d068b5f63428 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:27.040850 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ca013c-acbd-4a95-a36c-06c0a9dd2a74 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:27.068007 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:27.068721 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:27.094416 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83292c3-98f5-4437-b2b6-d50d60f985f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:27.095122 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83292c3-98f5-4437-b2b6-d50d60f985f3 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:27.095413 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83292c3-98f5-4437-b2b6-d50d60f985f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:27.095697 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83292c3-98f5-4437-b2b6-d50d60f985f3 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:27.096643 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bec6644-3331-457b-add9-26482ed4fafd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:27.096908 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83292c3-98f5-4437-b2b6-d50d60f985f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:27.097365 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83292c3-98f5-4437-b2b6-d50d60f985f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:27.097968 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bec6644-3331-457b-add9-26482ed4fafd None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:27.098401 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bec6644-3331-457b-add9-26482ed4fafd None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:27.098801 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83292c3-98f5-4437-b2b6-d50d60f985f3 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:27.098991 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bec6644-3331-457b-add9-26482ed4fafd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:27.112421 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bec6644-3331-457b-add9-26482ed4fafd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:27.114245 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bec6644-3331-457b-add9-26482ed4fafd None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:27.120005 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 395/794] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Jun 10 12:45:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:27.122145 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83292c3-98f5-4437-b2b6-d50d60f985f3 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:27.134572 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-0bec6644-3331-457b-add9-26482ed4fafd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:27.134572 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:27.134572 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:27.134572 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:27.134572 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:27.134572 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:27.134572 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:27.134572 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:27.134572 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:27.134572 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:27.134572 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:27.134572 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:27.134572 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 10 12:45:27.134572 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 10 12:45:27.134572 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 10 12:45:27.135545 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 10 12:45:27.135545 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:27.135545 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:27.135545 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:27.135545 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:27.135545 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:27.135545 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:27.135545 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:27.135545 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:27.135545 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:27.135545 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:27.135545 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:27.135545 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:27.135545 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:27.135545 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:27.135545 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:27.135545 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:27.136504 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 10 12:45:27.136504 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 10 12:45:27.136504 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:27.136504 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:27.139506 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 400/795] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:45:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:27.145321 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bec6644-3331-457b-add9-26482ed4fafd None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:27.252400 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83292c3-98f5-4437-b2b6-d50d60f985f3 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:27.255666 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bec6644-3331-457b-add9-26482ed4fafd None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:27.291709 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138c56ec-ba85-4af3-9189-6a7c10fa4d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:27.292765 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138c56ec-ba85-4af3-9189-6a7c10fa4d53 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:27.293252 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138c56ec-ba85-4af3-9189-6a7c10fa4d53 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:27.293753 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138c56ec-ba85-4af3-9189-6a7c10fa4d53 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:27.294677 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138c56ec-ba85-4af3-9189-6a7c10fa4d53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:27.295186 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138c56ec-ba85-4af3-9189-6a7c10fa4d53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:27.296806 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138c56ec-ba85-4af3-9189-6a7c10fa4d53 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:27.296987 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1161d8fc-5700-4747-b3fa-6dfe44aee3c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:27.298300 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1161d8fc-5700-4747-b3fa-6dfe44aee3c0 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:27.298902 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1161d8fc-5700-4747-b3fa-6dfe44aee3c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:27.299379 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1161d8fc-5700-4747-b3fa-6dfe44aee3c0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:27.309693 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1161d8fc-5700-4747-b3fa-6dfe44aee3c0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:27.311183 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1161d8fc-5700-4747-b3fa-6dfe44aee3c0 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:27.311420 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 401/796] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Jun 10 12:45:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 12:45:27.312930 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ed8ceb-8ea0-4411-9e7c-425991f49da6 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:27.313422 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ed8ceb-8ea0-4411-9e7c-425991f49da6 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:27.313897 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ed8ceb-8ea0-4411-9e7c-425991f49da6 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:27.315563 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 402/797] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:45:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:45:27.317014 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ed8ceb-8ea0-4411-9e7c-425991f49da6 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:27.331715 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-1161d8fc-5700-4747-b3fa-6dfe44aee3c0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 10 12:45:27.331715 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:27.331715 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:27.331715 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:27.331715 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:27.331715 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:27.331715 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:27.331715 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:27.331715 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:27.331715 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:27.331715 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:27.331715 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:27.331715 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 10 12:45:27.331715 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 10 12:45:27.331715 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 10 12:45:27.331715 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 10 12:45:27.332917 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:27.332917 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:27.332917 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 10 12:45:27.332917 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 10 12:45:27.332917 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 10 12:45:27.332917 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 10 12:45:27.332917 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:27.332917 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:27.332917 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:27.332917 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:27.332917 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:27.332917 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:27.332917 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:27.332917 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:27.332917 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:27.332917 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:27.332917 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:27.332917 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:27.334121 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:27.334121 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:27.334121 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 10 12:45:27.334121 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 10 12:45:27.334121 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 10 12:45:27.334121 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 10 12:45:27.334121 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 10 12:45:27.334121 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 10 12:45:27.334121 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 10 12:45:27.334121 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:27.338107 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 396/798] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Jun 10 12:45:27 2022] GET /identity/v3/users/swift => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:27.339456 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fbcf01-2cbe-4c05-805a-d68cfa08d890 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:27.340018 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fbcf01-2cbe-4c05-805a-d68cfa08d890 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:27.340176 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fbcf01-2cbe-4c05-805a-d68cfa08d890 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:27.347306 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:27.359731 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-19fbcf01-2cbe-4c05-805a-d68cfa08d890 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:45:27.360772 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-19fbcf01-2cbe-4c05-805a-d68cfa08d890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:45:27.389769 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:27.429543 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ed8ceb-8ea0-4411-9e7c-425991f49da6 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:27.439916 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 397/799] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:45:27 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 10 12:45:27.441121 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fbcf01-2cbe-4c05-805a-d68cfa08d890 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:27.465549 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9bdb71-7ea2-4175-b4d1-08f506d4473a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:27.467032 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9bdb71-7ea2-4175-b4d1-08f506d4473a None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:27.467553 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9bdb71-7ea2-4175-b4d1-08f506d4473a None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:27.468063 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9bdb71-7ea2-4175-b4d1-08f506d4473a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:27.476777 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9bdb71-7ea2-4175-b4d1-08f506d4473a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:27.478639 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9bdb71-7ea2-4175-b4d1-08f506d4473a None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:27.494114 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-3f9bdb71-7ea2-4175-b4d1-08f506d4473a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 10 12:45:27.494114 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:27.494114 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:27.494114 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:27.494114 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:27.494114 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:27.494114 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:27.494114 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:27.494114 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:27.494114 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:27.494114 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:27.494114 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:27.494114 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 10 12:45:27.494114 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 10 12:45:27.494114 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 10 12:45:27.495169 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 10 12:45:27.495169 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:27.495169 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:27.495169 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:27.495169 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:27.495169 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:27.495169 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:27.495169 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:27.495169 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:27.495169 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:27.495169 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:27.495169 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:27.495169 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:27.495169 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:27.495169 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:27.495169 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:27.495169 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:27.496115 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 10 12:45:27.496115 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 10 12:45:27.496115 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 10 12:45:27.496115 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 10 12:45:27.496115 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 10 12:45:27.496115 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 10 12:45:27.496115 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 10 12:45:27.496115 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:27.498647 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 403/800] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Jun 10 12:45:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:27.499751 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9bdb71-7ea2-4175-b4d1-08f506d4473a None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:27.562941 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fbcf01-2cbe-4c05-805a-d68cfa08d890 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:27.598681 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:27.598882 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54035ab4-368f-4557-855d-36d52ca5d3f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:27.599363 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54035ab4-368f-4557-855d-36d52ca5d3f9 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:27.599652 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54035ab4-368f-4557-855d-36d52ca5d3f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:27.599971 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54035ab4-368f-4557-855d-36d52ca5d3f9 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:27.601381 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54035ab4-368f-4557-855d-36d52ca5d3f9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:27.601698 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54035ab4-368f-4557-855d-36d52ca5d3f9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:27.603133 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54035ab4-368f-4557-855d-36d52ca5d3f9 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:27.607133 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9bdb71-7ea2-4175-b4d1-08f506d4473a None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:27.625811 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 398/801] 10.222.0.27 () {56 vars in 1135 bytes} [Fri Jun 10 12:45:27 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:27.626776 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54035ab4-368f-4557-855d-36d52ca5d3f9 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:27.643401 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0831cb3-b681-42e0-b01f-482ce7853c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:27.644570 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0831cb3-b681-42e0-b01f-482ce7853c1b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:27.645075 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0831cb3-b681-42e0-b01f-482ce7853c1b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:27.645547 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0831cb3-b681-42e0-b01f-482ce7853c1b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:27.646397 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0831cb3-b681-42e0-b01f-482ce7853c1b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:27.648509 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0831cb3-b681-42e0-b01f-482ce7853c1b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:27.656207 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:27.673952 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-f0831cb3-b681-42e0-b01f-482ce7853c1b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 10 12:45:27.673952 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:27.673952 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 10 12:45:27.673952 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 10 12:45:27.673952 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 10 12:45:27.673952 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 10 12:45:27.673952 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 10 12:45:27.673952 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 10 12:45:27.673952 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 10 12:45:27.673952 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:27.673952 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 10 12:45:27.673952 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:27.673952 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:27.673952 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:27.673952 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:27.673952 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:27.674917 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:27.674917 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:27.674917 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:27.674917 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:27.674917 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:27.674917 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:27.674917 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:27.674917 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 10 12:45:27.674917 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 10 12:45:27.674917 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 10 12:45:27.674917 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 10 12:45:27.674917 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:27.674917 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:27.674917 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:27.674917 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:27.674917 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:27.675791 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:27.675791 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:27.675791 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:27.675791 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:27.675791 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:27.675791 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:27.675791 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:27.675791 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:27.675791 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:27.675791 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:27.675791 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:27.675791 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 10 12:45:27.675791 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 10 12:45:27.675791 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 10 12:45:27.675791 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:27.676631 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 404/802] 10.222.0.27 () {56 vars in 1129 bytes} [Fri Jun 10 12:45:27 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:27.677510 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0831cb3-b681-42e0-b01f-482ce7853c1b None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:27.741318 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54035ab4-368f-4557-855d-36d52ca5d3f9 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:27.777178 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128d517f-2a15-4ce2-bb31-29197d7a1899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:27.777842 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128d517f-2a15-4ce2-bb31-29197d7a1899 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:27.778123 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128d517f-2a15-4ce2-bb31-29197d7a1899 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:27.778398 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128d517f-2a15-4ce2-bb31-29197d7a1899 None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:27.779184 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128d517f-2a15-4ce2-bb31-29197d7a1899 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:27.779490 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128d517f-2a15-4ce2-bb31-29197d7a1899 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:27.780946 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128d517f-2a15-4ce2-bb31-29197d7a1899 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:27.793386 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 399/803] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Jun 10 12:45:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:27.794238 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128d517f-2a15-4ce2-bb31-29197d7a1899 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:27.821650 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0831cb3-b681-42e0-b01f-482ce7853c1b None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:27.870904 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef8ed69-d0f6-447d-b13b-f53393dbcd15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:27.871707 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef8ed69-d0f6-447d-b13b-f53393dbcd15 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:27.872019 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef8ed69-d0f6-447d-b13b-f53393dbcd15 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:27.872296 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef8ed69-d0f6-447d-b13b-f53393dbcd15 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:27.885724 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef8ed69-d0f6-447d-b13b-f53393dbcd15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:27.887183 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef8ed69-d0f6-447d-b13b-f53393dbcd15 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:27.895625 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:27.905875 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-bef8ed69-d0f6-447d-b13b-f53393dbcd15 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:27.905875 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:27.905875 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:27.905875 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:27.905875 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:27.905875 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:27.905875 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:27.905875 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:27.905875 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:27.905875 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:27.905875 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:27.905875 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:27.905875 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 10 12:45:27.905875 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 10 12:45:27.905875 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 10 12:45:27.914426 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 10 12:45:27.914426 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:27.914426 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:27.914426 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:27.914426 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:27.914426 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:27.914426 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:27.914426 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:27.914426 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:27.914426 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:27.914426 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:27.914426 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:27.914426 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:27.914426 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:27.914426 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:27.914426 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:27.914426 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:27.915563 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 10 12:45:27.915563 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 10 12:45:27.915563 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:27.915563 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:27.915853 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:27.915964 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 405/804] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:45:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:27.915964 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef8ed69-d0f6-447d-b13b-f53393dbcd15 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:27.929924 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128d517f-2a15-4ce2-bb31-29197d7a1899 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:27.967618 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5487cec-1113-44f4-a5ce-afe3515c9e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:27.969251 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5487cec-1113-44f4-a5ce-afe3515c9e44 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:27.969372 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5487cec-1113-44f4-a5ce-afe3515c9e44 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:27.969741 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5487cec-1113-44f4-a5ce-afe3515c9e44 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:27.970539 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5487cec-1113-44f4-a5ce-afe3515c9e44 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:27.970945 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5487cec-1113-44f4-a5ce-afe3515c9e44 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:27.972711 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5487cec-1113-44f4-a5ce-afe3515c9e44 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:27.982201 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 400/805] 10.222.0.27 () {56 vars in 1124 bytes} [Fri Jun 10 12:45:27 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:27.983169 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5487cec-1113-44f4-a5ce-afe3515c9e44 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:28.026248 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef8ed69-d0f6-447d-b13b-f53393dbcd15 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:28.060976 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb846491-1011-4bc8-aae3-74cdb627c8f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:28.062250 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb846491-1011-4bc8-aae3-74cdb627c8f3 None admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:28.062721 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb846491-1011-4bc8-aae3-74cdb627c8f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:28.063176 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb846491-1011-4bc8-aae3-74cdb627c8f3 None admin] PATH_INFO: `/v3/projects/73ea5b23a2d34544ab493f4c5e4034b8/users/da66362070694e91bbe2cfbbd161b8a7/roles/ea7e8f74523b4322857fb3f4307d171e` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:28.080194 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb846491-1011-4bc8-aae3-74cdb627c8f3 None admin] RBAC: Authorizing `identity:create_grant(project_id=73ea5b23a2d34544ab493f4c5e4034b8, user_id=da66362070694e91bbe2cfbbd161b8a7, role_id=ea7e8f74523b4322857fb3f4307d171e)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:28.082053 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb846491-1011-4bc8-aae3-74cdb627c8f3 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:28.092484 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5487cec-1113-44f4-a5ce-afe3515c9e44 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:28.127771 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ac1fa4-2494-4750-964c-d3d3206135d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:28.128469 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ac1fa4-2494-4750-964c-d3d3206135d2 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:28.128752 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ac1fa4-2494-4750-964c-d3d3206135d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:28.129025 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ac1fa4-2494-4750-964c-d3d3206135d2 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:28.129792 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ac1fa4-2494-4750-964c-d3d3206135d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:28.130099 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ac1fa4-2494-4750-964c-d3d3206135d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:28.131473 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ac1fa4-2494-4750-964c-d3d3206135d2 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:28.144189 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 401/806] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Jun 10 12:45:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:28.148418 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ac1fa4-2494-4750-964c-d3d3206135d2 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:28.149827 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:28.160859 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 406/807] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Jun 10 12:45:27 2022] PUT /identity/v3/projects/73ea5b23a2d34544ab493f4c5e4034b8/users/da66362070694e91bbe2cfbbd161b8a7/roles/ea7e8f74523b4322857fb3f4307d171e => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 12:45:28.162551 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb846491-1011-4bc8-aae3-74cdb627c8f3 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:28.276770 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:28.354892 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ac1fa4-2494-4750-964c-d3d3206135d2 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:28.355134 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb846491-1011-4bc8-aae3-74cdb627c8f3 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:28.401522 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505bf553-b805-49dd-b382-0bd936ef0927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:28.402194 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505bf553-b805-49dd-b382-0bd936ef0927 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:28.402476 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505bf553-b805-49dd-b382-0bd936ef0927 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:28.402750 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505bf553-b805-49dd-b382-0bd936ef0927 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:28.403154 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fb33da-3114-4672-b99c-2681c73f460f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:28.403459 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505bf553-b805-49dd-b382-0bd936ef0927 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:28.403764 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505bf553-b805-49dd-b382-0bd936ef0927 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:28.404611 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fb33da-3114-4672-b99c-2681c73f460f None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:28.405125 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fb33da-3114-4672-b99c-2681c73f460f None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:28.405230 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505bf553-b805-49dd-b382-0bd936ef0927 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:28.405693 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fb33da-3114-4672-b99c-2681c73f460f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:28.411162 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 402/808] 10.222.0.27 () {56 vars in 1124 bytes} [Fri Jun 10 12:45:28 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:28.414387 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fb33da-3114-4672-b99c-2681c73f460f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:28.416321 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fb33da-3114-4672-b99c-2681c73f460f None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:28.417239 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505bf553-b805-49dd-b382-0bd936ef0927 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:28.433400 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-81fb33da-3114-4672-b99c-2681c73f460f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 10 12:45:28.433400 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:28.433400 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:28.433400 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:28.433400 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:28.433400 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:28.433400 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:28.433400 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:28.433400 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:28.433400 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:28.433400 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:28.433400 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:28.433400 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 10 12:45:28.433400 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 10 12:45:28.433400 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 10 12:45:28.434369 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 10 12:45:28.434369 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:28.434369 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:28.434369 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:28.434369 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:28.434369 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:28.434369 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:28.434369 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:28.434369 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:28.434369 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:28.434369 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:28.434369 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:28.434369 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:28.434369 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:28.434369 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:28.434369 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:28.434369 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:28.435404 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 10 12:45:28.435404 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 10 12:45:28.435404 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 10 12:45:28.435404 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 10 12:45:28.435404 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 10 12:45:28.435404 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 10 12:45:28.435404 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 10 12:45:28.435404 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:28.438086 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 407/809] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Jun 10 12:45:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:28.443260 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fb33da-3114-4672-b99c-2681c73f460f None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:28.499123 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:28.567147 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505bf553-b805-49dd-b382-0bd936ef0927 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:28.623093 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041645ed-003f-4334-bda0-d46d548b9e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:28.623760 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041645ed-003f-4334-bda0-d46d548b9e48 None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:28.624077 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041645ed-003f-4334-bda0-d46d548b9e48 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:28.624358 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041645ed-003f-4334-bda0-d46d548b9e48 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:28.624825 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fb33da-3114-4672-b99c-2681c73f460f None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:28.625283 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041645ed-003f-4334-bda0-d46d548b9e48 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:28.626458 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041645ed-003f-4334-bda0-d46d548b9e48 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:28.638594 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:28.663800 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9915592e-f90f-443e-839d-22b41e9d9f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:28.664552 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9915592e-f90f-443e-839d-22b41e9d9f40 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:28.664843 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9915592e-f90f-443e-839d-22b41e9d9f40 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:28.665119 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9915592e-f90f-443e-839d-22b41e9d9f40 None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:28.665912 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9915592e-f90f-443e-839d-22b41e9d9f40 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:28.666223 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9915592e-f90f-443e-839d-22b41e9d9f40 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:28.667658 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9915592e-f90f-443e-839d-22b41e9d9f40 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:28.680995 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 408/810] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Jun 10 12:45:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 12:45:28.682224 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efa3b7a-f75e-4197-8222-a32c9be744e6 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:28.682514 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efa3b7a-f75e-4197-8222-a32c9be744e6 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:28.682789 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efa3b7a-f75e-4197-8222-a32c9be744e6 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:28.687909 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 409/811] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:45:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:45:28.687909 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efa3b7a-f75e-4197-8222-a32c9be744e6 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:28.743594 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 403/812] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Jun 10 12:45:28 2022] POST /identity/v3/endpoints => generated 346 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 10 12:45:28.745684 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b26951-fd56-49df-9bff-09b90b549425 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:28.746181 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b26951-fd56-49df-9bff-09b90b549425 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:28.746701 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b26951-fd56-49df-9bff-09b90b549425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:28.768881 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-01b26951-fd56-49df-9bff-09b90b549425 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:45:28.770440 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-01b26951-fd56-49df-9bff-09b90b549425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:45:28.861160 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 404/813] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:45:28 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 10 12:45:28.885483 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efa3b7a-f75e-4197-8222-a32c9be744e6 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:28.890994 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b26951-fd56-49df-9bff-09b90b549425 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:28.948268 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:28.981214 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b61370d-3f23-4706-bbe4-3470ae3cf67a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:28.982138 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b61370d-3f23-4706-bbe4-3470ae3cf67a None admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:28.982781 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b61370d-3f23-4706-bbe4-3470ae3cf67a None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:28.982781 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b61370d-3f23-4706-bbe4-3470ae3cf67a None admin] PATH_INFO: `/v3/projects/73ea5b23a2d34544ab493f4c5e4034b8/users/fb63b9cd74bf438aa25b72cef8337fec/roles/b43d5a8247e244869c2c70f986f42e8a` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:28.989324 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b61370d-3f23-4706-bbe4-3470ae3cf67a None admin] RBAC: Authorizing `identity:create_grant(project_id=73ea5b23a2d34544ab493f4c5e4034b8, user_id=fb63b9cd74bf438aa25b72cef8337fec, role_id=b43d5a8247e244869c2c70f986f42e8a)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:28.990803 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b61370d-3f23-4706-bbe4-3470ae3cf67a None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:29.003196 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b26951-fd56-49df-9bff-09b90b549425 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:29.040660 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b663471-de8c-434d-ba20-4a2e29bb213b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:29.041470 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b663471-de8c-434d-ba20-4a2e29bb213b None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:29.041754 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b663471-de8c-434d-ba20-4a2e29bb213b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:29.042032 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b663471-de8c-434d-ba20-4a2e29bb213b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:29.052855 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b663471-de8c-434d-ba20-4a2e29bb213b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:45:29.061617 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-1b663471-de8c-434d-ba20-4a2e29bb213b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 10 12:45:29.061617 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:29.061617 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:29.061617 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:29.061617 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:29.061617 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:29.061617 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:29.061617 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:29.061617 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:29.061617 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:29.061617 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:29.061617 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:29.061617 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:45:29.061617 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:45:29.061617 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:45:29.061617 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:45:29.068613 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:45:29.068613 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:45:29.068613 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:45:29.068613 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:45:29.068613 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:29.068613 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:29.068613 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:29.068613 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:29.068613 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:29.068613 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:29.068613 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:29.068613 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:29.068613 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:29.068613 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:29.068613 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:29.068613 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:29.069542 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:29.069542 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:29.069542 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:29.069542 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:29.069542 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:45:29.069542 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:45:29.069542 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:45:29.069542 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:45:29.069542 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:45:29.069542 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:45:29.069542 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 10 12:45:29.069542 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:29.069542 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 405/814] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Jun 10 12:45:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 10 12:45:29.069542 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b663471-de8c-434d-ba20-4a2e29bb213b None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:29.094733 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 410/815] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Jun 10 12:45:28 2022] PUT /identity/v3/projects/73ea5b23a2d34544ab493f4c5e4034b8/users/fb63b9cd74bf438aa25b72cef8337fec/roles/b43d5a8247e244869c2c70f986f42e8a => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 10 12:45:29.202184 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b663471-de8c-434d-ba20-4a2e29bb213b None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:29.216593 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:29.248466 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6fa6b4-d3f4-4397-8121-dae00aa26a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:29.249170 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6fa6b4-d3f4-4397-8121-dae00aa26a13 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:29.249455 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6fa6b4-d3f4-4397-8121-dae00aa26a13 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:29.249731 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6fa6b4-d3f4-4397-8121-dae00aa26a13 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:29.250509 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6fa6b4-d3f4-4397-8121-dae00aa26a13 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:29.250818 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6fa6b4-d3f4-4397-8121-dae00aa26a13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:29.252290 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6fa6b4-d3f4-4397-8121-dae00aa26a13 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:29.262782 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 406/816] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Jun 10 12:45:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 12:45:29.268426 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b61370d-3f23-4706-bbe4-3470ae3cf67a None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:29.320326 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:29.379819 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b61370d-3f23-4706-bbe4-3470ae3cf67a None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:29.399220 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97474a96-f525-43e2-9bc6-10cf52908953 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:29.399528 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97474a96-f525-43e2-9bc6-10cf52908953 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:29.399777 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97474a96-f525-43e2-9bc6-10cf52908953 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:29.401455 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 407/817] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:45:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:45:29.407870 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eb6ce4-d6d3-492c-be9e-e65f69448acd None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:29.408177 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eb6ce4-d6d3-492c-be9e-e65f69448acd None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:29.408463 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eb6ce4-d6d3-492c-be9e-e65f69448acd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:29.428527 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-e6eb6ce4-d6d3-492c-be9e-e65f69448acd None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:45:29.429218 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79c4dcc-dd58-4d1a-93f6-33db0c43d732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:29.429727 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e6eb6ce4-d6d3-492c-be9e-e65f69448acd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:45:29.430030 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79c4dcc-dd58-4d1a-93f6-33db0c43d732 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:29.430319 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79c4dcc-dd58-4d1a-93f6-33db0c43d732 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:29.430590 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79c4dcc-dd58-4d1a-93f6-33db0c43d732 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:29.443557 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79c4dcc-dd58-4d1a-93f6-33db0c43d732 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:29.445916 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79c4dcc-dd58-4d1a-93f6-33db0c43d732 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:29.471192 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-b79c4dcc-dd58-4d1a-93f6-33db0c43d732 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:29.471192 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:29.471192 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:29.471192 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:29.471192 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:29.471192 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:29.471192 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:29.471192 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:29.471192 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:29.471192 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:29.471192 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:29.471192 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:29.471192 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 10 12:45:29.471192 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 10 12:45:29.471192 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 10 12:45:29.472176 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 10 12:45:29.472176 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:29.472176 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:29.472176 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:29.472176 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:29.472176 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:29.472176 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:29.472176 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:29.472176 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:29.472176 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:29.472176 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:29.472176 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:29.472176 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:29.472176 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:29.472176 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:29.472176 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:29.472176 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:29.473520 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 10 12:45:29.473520 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 10 12:45:29.473520 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:29.473520 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:29.473520 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 411/818] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:45:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:29.478365 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79c4dcc-dd58-4d1a-93f6-33db0c43d732 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:29.521475 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 408/819] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:45:29 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 10 12:45:29.545166 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eb6ce4-d6d3-492c-be9e-e65f69448acd None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:29.572891 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:29.603150 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79c4dcc-dd58-4d1a-93f6-33db0c43d732 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:29.669496 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853dfeba-e538-4931-87ba-4544a792c277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:29.671684 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853dfeba-e538-4931-87ba-4544a792c277 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:29.676259 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853dfeba-e538-4931-87ba-4544a792c277 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:29.676757 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853dfeba-e538-4931-87ba-4544a792c277 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:29.677783 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853dfeba-e538-4931-87ba-4544a792c277 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:29.678309 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853dfeba-e538-4931-87ba-4544a792c277 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:29.680476 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853dfeba-e538-4931-87ba-4544a792c277 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:29.701966 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 412/820] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Jun 10 12:45:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 12:45:29.710944 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853dfeba-e538-4931-87ba-4544a792c277 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:29.731283 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eb6ce4-d6d3-492c-be9e-e65f69448acd None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:29.740307 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:29.775337 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6711600c-ab87-4aa2-a34e-5ea3e8564a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:29.776195 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6711600c-ab87-4aa2-a34e-5ea3e8564a66 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:29.776482 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6711600c-ab87-4aa2-a34e-5ea3e8564a66 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:29.776759 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6711600c-ab87-4aa2-a34e-5ea3e8564a66 None admin] PATH_INFO: `/v3/services/network` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:29.777420 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6711600c-ab87-4aa2-a34e-5ea3e8564a66 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:29.778846 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6711600c-ab87-4aa2-a34e-5ea3e8564a66 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:29.792190 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-6711600c-ab87-4aa2-a34e-5ea3e8564a66 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 10 12:45:29.792190 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:29.792190 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 10 12:45:29.792190 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 10 12:45:29.792190 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 10 12:45:29.792190 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 10 12:45:29.792190 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 10 12:45:29.792190 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 10 12:45:29.792190 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 10 12:45:29.792190 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:29.792190 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 10 12:45:29.792190 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:29.792190 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:29.792190 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:29.792190 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:29.792190 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:29.792190 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:29.793225 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:29.793225 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:29.793225 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:29.793225 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:29.793225 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:29.793225 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:29.793225 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 10 12:45:29.793225 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 10 12:45:29.793225 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 10 12:45:29.793225 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 10 12:45:29.793225 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:29.793225 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:29.793225 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:29.793225 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:29.793225 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:29.793225 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:29.794173 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:29.794173 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:29.794173 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:29.794173 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:29.794173 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:29.794173 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:29.794173 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:29.794173 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:29.794173 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:29.794173 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:29.794173 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 10 12:45:29.794173 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 10 12:45:29.794173 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 10 12:45:29.794173 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:29.794173 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 409/821] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Jun 10 12:45:29 2022] GET /identity/v3/services/network => generated 88 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:29.799253 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6711600c-ab87-4aa2-a34e-5ea3e8564a66 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:29.894499 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853dfeba-e538-4931-87ba-4544a792c277 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:29.955884 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a346f7-e183-4aef-a44f-621c414e9f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:29.957606 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a346f7-e183-4aef-a44f-621c414e9f79 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:29.958192 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a346f7-e183-4aef-a44f-621c414e9f79 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:29.959533 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a346f7-e183-4aef-a44f-621c414e9f79 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:29.970493 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a346f7-e183-4aef-a44f-621c414e9f79 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:29.972661 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6711600c-ab87-4aa2-a34e-5ea3e8564a66 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:29.977371 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a346f7-e183-4aef-a44f-621c414e9f79 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:29.999918 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-a9a346f7-e183-4aef-a44f-621c414e9f79 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 10 12:45:29.999918 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:29.999918 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:29.999918 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:29.999918 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:29.999918 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:29.999918 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:29.999918 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:29.999918 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:29.999918 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:29.999918 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:29.999918 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:29.999918 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 10 12:45:29.999918 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 10 12:45:29.999918 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 10 12:45:29.999918 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 10 12:45:30.000924 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:30.000924 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:30.000924 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 10 12:45:30.000924 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 10 12:45:30.000924 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 10 12:45:30.000924 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 10 12:45:30.000924 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:30.000924 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:30.000924 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:30.000924 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:30.000924 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:30.000924 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:30.000924 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:30.000924 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:30.000924 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:30.000924 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:30.000924 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:30.000924 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:30.001885 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:30.001885 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:30.001885 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 10 12:45:30.001885 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 10 12:45:30.001885 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 10 12:45:30.001885 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 10 12:45:30.001885 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 10 12:45:30.001885 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 10 12:45:30.001885 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 10 12:45:30.001885 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:30.001885 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 413/822] 10.222.0.27 () {56 vars in 1093 bytes} [Fri Jun 10 12:45:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:30.006685 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a346f7-e183-4aef-a44f-621c414e9f79 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:30.012166 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:30.023074 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadb2ebd-83d1-4fd1-994f-9b4a72f51adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:30.023751 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadb2ebd-83d1-4fd1-994f-9b4a72f51adf None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:30.024061 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadb2ebd-83d1-4fd1-994f-9b4a72f51adf None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:30.024342 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadb2ebd-83d1-4fd1-994f-9b4a72f51adf None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:30.025061 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadb2ebd-83d1-4fd1-994f-9b4a72f51adf None admin] RBAC: Adding query filter params (name=network) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:30.025425 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadb2ebd-83d1-4fd1-994f-9b4a72f51adf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:30.029052 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadb2ebd-83d1-4fd1-994f-9b4a72f51adf None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:30.037223 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 410/823] 10.222.0.27 () {56 vars in 1112 bytes} [Fri Jun 10 12:45:29 2022] GET /identity/v3/services?name=network => generated 124 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 12:45:30.043235 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadb2ebd-83d1-4fd1-994f-9b4a72f51adf None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:30.105499 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:30.195492 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a346f7-e183-4aef-a44f-621c414e9f79 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:30.231067 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68920089-5d90-43f4-a1d1-ed06d47d314e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:30.231748 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68920089-5d90-43f4-a1d1-ed06d47d314e None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:30.232063 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68920089-5d90-43f4-a1d1-ed06d47d314e None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:30.232345 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68920089-5d90-43f4-a1d1-ed06d47d314e None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:30.233191 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68920089-5d90-43f4-a1d1-ed06d47d314e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:30.233497 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68920089-5d90-43f4-a1d1-ed06d47d314e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:30.234939 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68920089-5d90-43f4-a1d1-ed06d47d314e None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:30.253877 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 414/824] 10.222.0.27 () {56 vars in 1133 bytes} [Fri Jun 10 12:45:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:30.259219 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68920089-5d90-43f4-a1d1-ed06d47d314e None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:30.274733 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadb2ebd-83d1-4fd1-994f-9b4a72f51adf None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:30.336292 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:30.369474 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68920089-5d90-43f4-a1d1-ed06d47d314e None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:30.394425 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f40944-de03-4096-bb51-1a758c6a79d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:30.395139 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f40944-de03-4096-bb51-1a758c6a79d0 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:30.395430 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f40944-de03-4096-bb51-1a758c6a79d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:30.395707 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f40944-de03-4096-bb51-1a758c6a79d0 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:30.396454 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f40944-de03-4096-bb51-1a758c6a79d0 None admin] RBAC: Adding query filter params (type=network) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:30.396768 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f40944-de03-4096-bb51-1a758c6a79d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:30.398678 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f40944-de03-4096-bb51-1a758c6a79d0 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:30.409074 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 411/825] 10.222.0.27 () {56 vars in 1112 bytes} [Fri Jun 10 12:45:30 2022] GET /identity/v3/services?type=network => generated 350 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 12:45:30.411799 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32080a7d-face-4236-ba96-997638a5b4b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:30.412650 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32080a7d-face-4236-ba96-997638a5b4b1 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:30.413478 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32080a7d-face-4236-ba96-997638a5b4b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:30.413756 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32080a7d-face-4236-ba96-997638a5b4b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:30.415165 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f40944-de03-4096-bb51-1a758c6a79d0 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:30.526249 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32080a7d-face-4236-ba96-997638a5b4b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:30.527630 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32080a7d-face-4236-ba96-997638a5b4b1 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:30.596967 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:30.606510 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:30.623726 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f40944-de03-4096-bb51-1a758c6a79d0 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:30.645483 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 415/826] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Jun 10 12:45:30 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 10 12:45:30.653127 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32080a7d-face-4236-ba96-997638a5b4b1 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:30.690074 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3b6f3b-47c6-4d8f-bb6c-87bd19acc5f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:30.690766 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3b6f3b-47c6-4d8f-bb6c-87bd19acc5f6 None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:30.691052 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3b6f3b-47c6-4d8f-bb6c-87bd19acc5f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:30.691335 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3b6f3b-47c6-4d8f-bb6c-87bd19acc5f6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:30.692140 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3b6f3b-47c6-4d8f-bb6c-87bd19acc5f6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:30.693574 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3b6f3b-47c6-4d8f-bb6c-87bd19acc5f6 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:30.778717 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 412/827] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Jun 10 12:45:30 2022] POST /identity/v3/endpoints => generated 327 bytes in 364 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 12:45:30.779995 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290dffcc-cc10-46b4-9273-2d38739ff92f None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:30.780291 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290dffcc-cc10-46b4-9273-2d38739ff92f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:30.781002 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290dffcc-cc10-46b4-9273-2d38739ff92f None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:30.782101 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 413/828] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:45:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:45:30.789370 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e01d79-65f5-486c-8328-c9a5423f2334 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:30.789681 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e01d79-65f5-486c-8328-c9a5423f2334 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:30.790529 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e01d79-65f5-486c-8328-c9a5423f2334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:30.796918 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32080a7d-face-4236-ba96-997638a5b4b1 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:30.813983 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-79e01d79-65f5-486c-8328-c9a5423f2334 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:45:30.816092 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-79e01d79-65f5-486c-8328-c9a5423f2334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:45:30.857683 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14245485-312d-46d9-8e11-e5b7f51d4764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:30.858871 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14245485-312d-46d9-8e11-e5b7f51d4764 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:30.859400 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14245485-312d-46d9-8e11-e5b7f51d4764 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:30.859905 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14245485-312d-46d9-8e11-e5b7f51d4764 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:30.884729 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14245485-312d-46d9-8e11-e5b7f51d4764 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:30.886210 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14245485-312d-46d9-8e11-e5b7f51d4764 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:30.908112 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 414/829] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:45:30 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 10 12:45:30.931397 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e01d79-65f5-486c-8328-c9a5423f2334 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:30.937432 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-14245485-312d-46d9-8e11-e5b7f51d4764 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:30.937432 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:30.937432 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:30.937432 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:30.937432 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:30.937432 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:30.937432 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:30.937432 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:30.937432 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:30.937432 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:30.937432 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:30.937432 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:30.937432 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 10 12:45:30.937432 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 10 12:45:30.937432 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 10 12:45:30.938385 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 10 12:45:30.938385 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:30.938385 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:30.938385 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:30.938385 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:30.938385 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:30.938385 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:30.938385 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:30.938385 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:30.938385 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:30.938385 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:30.938385 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:30.938385 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:30.938385 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:30.938385 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:30.938385 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:30.938385 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:30.939299 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 10 12:45:30.939299 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 10 12:45:30.939299 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:30.939299 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:30.939299 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 416/830] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:45:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:30.944509 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14245485-312d-46d9-8e11-e5b7f51d4764 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:30.961115 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:31.068564 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14245485-312d-46d9-8e11-e5b7f51d4764 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:31.110674 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcea3830-38e2-4314-9da5-3f9888044d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:31.111447 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcea3830-38e2-4314-9da5-3f9888044d4e None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:31.111744 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcea3830-38e2-4314-9da5-3f9888044d4e None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:31.112151 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcea3830-38e2-4314-9da5-3f9888044d4e None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:31.112893 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcea3830-38e2-4314-9da5-3f9888044d4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:31.113203 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcea3830-38e2-4314-9da5-3f9888044d4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:31.114680 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcea3830-38e2-4314-9da5-3f9888044d4e None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:31.127952 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 417/831] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Jun 10 12:45:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 12:45:31.133132 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcea3830-38e2-4314-9da5-3f9888044d4e None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:31.153999 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:31.179616 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e01d79-65f5-486c-8328-c9a5423f2334 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:31.227746 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:31.228297 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b48cc3-bb82-433d-a21f-309f94ea1d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:31.229358 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b48cc3-bb82-433d-a21f-309f94ea1d73 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:31.229972 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b48cc3-bb82-433d-a21f-309f94ea1d73 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:31.230553 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b48cc3-bb82-433d-a21f-309f94ea1d73 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:31.247409 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b48cc3-bb82-433d-a21f-309f94ea1d73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:45:31.255325 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-a4b48cc3-bb82-433d-a21f-309f94ea1d73 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 10 12:45:31.255325 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:31.255325 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:31.255325 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:31.255325 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:31.255325 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:31.255325 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:31.255325 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:31.255325 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:31.255325 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:31.255325 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:31.255325 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:31.255325 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:45:31.255325 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:45:31.255325 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:45:31.255325 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:45:31.256897 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:45:31.256897 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:45:31.256897 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:45:31.256897 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:45:31.256897 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:31.256897 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:31.256897 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:31.256897 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:31.256897 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:31.256897 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:31.256897 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:31.256897 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:31.256897 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:31.256897 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:31.256897 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:31.256897 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:31.257932 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:31.257932 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:31.257932 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:31.257932 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:31.257932 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:45:31.257932 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:45:31.257932 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:45:31.257932 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:45:31.257932 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:45:31.257932 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:45:31.257932 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 10 12:45:31.257932 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:31.261874 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 415/832] 10.222.0.27 () {56 vars in 1102 bytes} [Fri Jun 10 12:45:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:31.262711 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b48cc3-bb82-433d-a21f-309f94ea1d73 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:31.274500 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcea3830-38e2-4314-9da5-3f9888044d4e None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:31.316582 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d827d4a9-593f-4e2c-adaf-d78fb75af002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:31.317443 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d827d4a9-593f-4e2c-adaf-d78fb75af002 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:31.317734 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d827d4a9-593f-4e2c-adaf-d78fb75af002 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:31.318015 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d827d4a9-593f-4e2c-adaf-d78fb75af002 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:31.325179 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d827d4a9-593f-4e2c-adaf-d78fb75af002 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:31.326833 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d827d4a9-593f-4e2c-adaf-d78fb75af002 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:31.342970 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-d827d4a9-593f-4e2c-adaf-d78fb75af002 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 10 12:45:31.342970 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:31.342970 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:31.342970 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:31.342970 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:31.342970 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:31.342970 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:31.342970 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:31.342970 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:31.342970 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:31.342970 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:31.342970 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:31.342970 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 10 12:45:31.342970 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 10 12:45:31.342970 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 10 12:45:31.344965 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 10 12:45:31.344965 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:31.344965 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:31.344965 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:31.344965 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:31.344965 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:31.344965 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:31.344965 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:31.344965 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:31.344965 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:31.344965 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:31.344965 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:31.344965 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:31.344965 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:31.344965 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:31.344965 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:31.344965 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:31.346067 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 10 12:45:31.346067 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 10 12:45:31.346067 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 10 12:45:31.346067 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 10 12:45:31.346067 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 10 12:45:31.346067 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 10 12:45:31.346067 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 10 12:45:31.346067 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:31.346067 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 418/833] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Jun 10 12:45:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 10 12:45:31.350030 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d827d4a9-593f-4e2c-adaf-d78fb75af002 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:31.394439 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b48cc3-bb82-433d-a21f-309f94ea1d73 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:31.438113 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c5ee4c-adbb-481b-a5b1-3442d6a9ce4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:31.439156 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c5ee4c-adbb-481b-a5b1-3442d6a9ce4d None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:31.439156 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c5ee4c-adbb-481b-a5b1-3442d6a9ce4d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:31.439401 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c5ee4c-adbb-481b-a5b1-3442d6a9ce4d None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:31.440176 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c5ee4c-adbb-481b-a5b1-3442d6a9ce4d None admin] RBAC: Adding query filter params (name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:31.440478 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c5ee4c-adbb-481b-a5b1-3442d6a9ce4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:31.441845 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c5ee4c-adbb-481b-a5b1-3442d6a9ce4d None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:31.450358 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 416/834] 10.222.0.27 () {56 vars in 1103 bytes} [Fri Jun 10 12:45:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:31.451675 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f27b3b-c931-4f23-8c4d-bdc4820fe26a None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:31.451988 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f27b3b-c931-4f23-8c4d-bdc4820fe26a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:31.452273 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f27b3b-c931-4f23-8c4d-bdc4820fe26a None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:31.456417 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 417/835] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:45:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:45:31.456417 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f27b3b-c931-4f23-8c4d-bdc4820fe26a None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:31.459419 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:31.478126 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d827d4a9-593f-4e2c-adaf-d78fb75af002 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:31.505313 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:31.517890 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f39cf21-f960-4585-8396-f7ae649a90d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:31.518778 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f39cf21-f960-4585-8396-f7ae649a90d3 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:31.519794 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f39cf21-f960-4585-8396-f7ae649a90d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:31.520184 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f39cf21-f960-4585-8396-f7ae649a90d3 None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:31.520991 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f39cf21-f960-4585-8396-f7ae649a90d3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:31.521351 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f39cf21-f960-4585-8396-f7ae649a90d3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:31.523830 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f39cf21-f960-4585-8396-f7ae649a90d3 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:31.541684 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 419/836] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Jun 10 12:45:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:31.543066 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a294063-d915-4e2f-b998-9fab767ca1cd None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:31.543356 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a294063-d915-4e2f-b998-9fab767ca1cd None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:31.543633 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a294063-d915-4e2f-b998-9fab767ca1cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:31.561362 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-3a294063-d915-4e2f-b998-9fab767ca1cd None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:45:31.562770 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-3a294063-d915-4e2f-b998-9fab767ca1cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:45:31.603510 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f27b3b-c931-4f23-8c4d-bdc4820fe26a None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:31.641486 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 420/837] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:45:31 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 10 12:45:31.648364 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a294063-d915-4e2f-b998-9fab767ca1cd None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:31.650206 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeacb74d-8317-4312-850c-43a0a08d9eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:31.650393 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeacb74d-8317-4312-850c-43a0a08d9eee None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:31.650687 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeacb74d-8317-4312-850c-43a0a08d9eee None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:31.650972 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeacb74d-8317-4312-850c-43a0a08d9eee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:31.677895 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeacb74d-8317-4312-850c-43a0a08d9eee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:31.679392 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeacb74d-8317-4312-850c-43a0a08d9eee None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:31.699518 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-eeacb74d-8317-4312-850c-43a0a08d9eee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:31.699518 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:31.699518 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:31.699518 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:31.699518 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:31.699518 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:31.699518 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:31.699518 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:31.699518 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:31.699518 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:31.699518 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:31.699518 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:31.699518 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 10 12:45:31.699518 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 10 12:45:31.699518 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 10 12:45:31.700515 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 10 12:45:31.700515 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:31.700515 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:31.700515 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:31.700515 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:31.700515 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:31.700515 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:31.700515 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:31.700515 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:31.700515 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:31.700515 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:31.700515 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:31.700515 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:31.700515 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:31.700515 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:31.700515 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:31.700515 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:31.701417 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 10 12:45:31.701417 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 10 12:45:31.701417 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:31.701417 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:31.703642 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 418/838] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:45:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:31.704712 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeacb74d-8317-4312-850c-43a0a08d9eee None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:31.770393 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:31.792827 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a294063-d915-4e2f-b998-9fab767ca1cd None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:31.813294 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:31.827070 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeacb74d-8317-4312-850c-43a0a08d9eee None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:31.834479 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94f4dfe-8274-4096-90c0-94596bafd009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:31.835969 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94f4dfe-8274-4096-90c0-94596bafd009 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:31.836554 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94f4dfe-8274-4096-90c0-94596bafd009 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:31.836981 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94f4dfe-8274-4096-90c0-94596bafd009 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:31.837973 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94f4dfe-8274-4096-90c0-94596bafd009 None admin] RBAC: Adding query filter params (role.id=b43d5a8247e244869c2c70f986f42e8a, scope.project.id=73ea5b23a2d34544ab493f4c5e4034b8, user.id=8bd6caa461e84100a3d5fe5af5d80f3e) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:31.838487 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94f4dfe-8274-4096-90c0-94596bafd009 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:31.840171 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94f4dfe-8274-4096-90c0-94596bafd009 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:31.846841 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 421/839] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Jun 10 12:45:31 2022] GET /identity/v3/role_assignments?user.id=8bd6caa461e84100a3d5fe5af5d80f3e&scope.project.id=73ea5b23a2d34544ab493f4c5e4034b8&role.id=b43d5a8247e244869c2c70f986f42e8a => generated 608 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:31.847832 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94f4dfe-8274-4096-90c0-94596bafd009 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:31.883367 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3f3f98-ca53-4540-91d4-1964b003bb9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:31.884657 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3f3f98-ca53-4540-91d4-1964b003bb9c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:31.885110 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3f3f98-ca53-4540-91d4-1964b003bb9c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:31.885537 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3f3f98-ca53-4540-91d4-1964b003bb9c None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:31.886357 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3f3f98-ca53-4540-91d4-1964b003bb9c None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:31.888094 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3f3f98-ca53-4540-91d4-1964b003bb9c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:31.905519 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-3f3f3f98-ca53-4540-91d4-1964b003bb9c None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 10 12:45:31.905519 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:31.905519 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 10 12:45:31.905519 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 10 12:45:31.905519 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 10 12:45:31.905519 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 10 12:45:31.905519 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 10 12:45:31.905519 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 10 12:45:31.905519 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 10 12:45:31.905519 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:31.905519 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 10 12:45:31.905519 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:31.905519 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:31.905519 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:31.905519 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:31.905519 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:31.905519 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:31.906645 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:31.906645 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:31.906645 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:31.906645 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:31.906645 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:31.906645 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:31.906645 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 10 12:45:31.906645 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 10 12:45:31.906645 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 10 12:45:31.906645 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 10 12:45:31.906645 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:31.906645 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:31.906645 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:31.906645 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:31.906645 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:31.906645 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:31.907534 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:31.907534 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:31.907534 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:31.907534 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:31.907534 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:31.907534 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:31.907534 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:31.907534 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:31.907534 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:31.907534 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:31.907534 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 10 12:45:31.907534 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 10 12:45:31.907534 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 10 12:45:31.907534 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:31.907534 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 419/840] 10.222.0.27 () {56 vars in 1114 bytes} [Fri Jun 10 12:45:31 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:31.909424 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a046a6d-c322-4881-904c-f05aac65c615 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:31.909806 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a046a6d-c322-4881-904c-f05aac65c615 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:31.910807 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a046a6d-c322-4881-904c-f05aac65c615 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:31.916686 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 420/841] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:45:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:45:31.916686 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a046a6d-c322-4881-904c-f05aac65c615 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:32.003046 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94f4dfe-8274-4096-90c0-94596bafd009 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:32.043323 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b945e1f7-8dcb-4d56-98b2-0c49fd4c1218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:32.045030 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b945e1f7-8dcb-4d56-98b2-0c49fd4c1218 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:32.045030 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b945e1f7-8dcb-4d56-98b2-0c49fd4c1218 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:32.045030 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b945e1f7-8dcb-4d56-98b2-0c49fd4c1218 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:32.045373 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b945e1f7-8dcb-4d56-98b2-0c49fd4c1218 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:32.045454 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a046a6d-c322-4881-904c-f05aac65c615 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:32.045663 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b945e1f7-8dcb-4d56-98b2-0c49fd4c1218 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:32.047502 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b945e1f7-8dcb-4d56-98b2-0c49fd4c1218 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:32.062419 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 422/842] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Jun 10 12:45:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:32.063672 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655b630f-af27-41a4-adb5-d6e487e5dfcc None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:32.064647 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:32.064828 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655b630f-af27-41a4-adb5-d6e487e5dfcc None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:32.065682 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655b630f-af27-41a4-adb5-d6e487e5dfcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:32.082856 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b5a6c1-9839-4a8b-b42a-52e3af08eb64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:32.083301 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-655b630f-af27-41a4-adb5-d6e487e5dfcc None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:45:32.083517 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b5a6c1-9839-4a8b-b42a-52e3af08eb64 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:32.083798 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b5a6c1-9839-4a8b-b42a-52e3af08eb64 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:32.084140 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b5a6c1-9839-4a8b-b42a-52e3af08eb64 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:32.084574 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-655b630f-af27-41a4-adb5-d6e487e5dfcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:45:32.084877 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b5a6c1-9839-4a8b-b42a-52e3af08eb64 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:32.085183 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b5a6c1-9839-4a8b-b42a-52e3af08eb64 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:32.086531 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b5a6c1-9839-4a8b-b42a-52e3af08eb64 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:32.092270 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 421/843] 10.222.0.27 () {56 vars in 1114 bytes} [Fri Jun 10 12:45:31 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:32.093563 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b5a6c1-9839-4a8b-b42a-52e3af08eb64 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:32.122638 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:32.153428 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 423/844] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:45:32 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 10 12:45:32.160328 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655b630f-af27-41a4-adb5-d6e487e5dfcc None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:32.206775 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b5a6c1-9839-4a8b-b42a-52e3af08eb64 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:32.242918 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c560fd9b-a9f4-4fea-a825-40b2341e43ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:32.243721 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c560fd9b-a9f4-4fea-a825-40b2341e43ba None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:32.244034 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c560fd9b-a9f4-4fea-a825-40b2341e43ba None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:32.244314 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c560fd9b-a9f4-4fea-a825-40b2341e43ba None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:32.251768 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c560fd9b-a9f4-4fea-a825-40b2341e43ba None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:32.253223 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c560fd9b-a9f4-4fea-a825-40b2341e43ba None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:32.261001 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655b630f-af27-41a4-adb5-d6e487e5dfcc None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:32.267825 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-c560fd9b-a9f4-4fea-a825-40b2341e43ba None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 10 12:45:32.267825 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:32.267825 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:32.267825 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:32.267825 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:32.267825 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:32.267825 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:32.267825 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:32.267825 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:32.267825 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:32.267825 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:32.267825 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:32.267825 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 10 12:45:32.267825 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 10 12:45:32.267825 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 10 12:45:32.267825 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 10 12:45:32.268815 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:32.268815 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:32.268815 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 10 12:45:32.268815 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 10 12:45:32.268815 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 10 12:45:32.268815 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 10 12:45:32.268815 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:32.268815 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:32.268815 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:32.268815 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:32.268815 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:32.268815 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:32.268815 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:32.268815 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:32.268815 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:32.268815 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:32.268815 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:32.268815 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:32.269791 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:32.269791 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:32.269791 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 10 12:45:32.269791 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 10 12:45:32.269791 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 10 12:45:32.269791 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 10 12:45:32.269791 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 10 12:45:32.269791 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 10 12:45:32.269791 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 10 12:45:32.269791 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:32.269791 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 422/845] 10.222.0.27 () {56 vars in 1117 bytes} [Fri Jun 10 12:45:32 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:32.269791 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c560fd9b-a9f4-4fea-a825-40b2341e43ba None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:32.300525 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d0b7be-62f2-48e7-b85a-2713c7cea4ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:32.301189 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d0b7be-62f2-48e7-b85a-2713c7cea4ac None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:32.301470 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d0b7be-62f2-48e7-b85a-2713c7cea4ac None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:32.301749 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d0b7be-62f2-48e7-b85a-2713c7cea4ac None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:32.302446 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d0b7be-62f2-48e7-b85a-2713c7cea4ac None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:32.302747 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d0b7be-62f2-48e7-b85a-2713c7cea4ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:32.304685 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d0b7be-62f2-48e7-b85a-2713c7cea4ac None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:32.309332 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 424/846] 10.222.0.27 () {56 vars in 1114 bytes} [Fri Jun 10 12:45:32 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:32.309934 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d0b7be-62f2-48e7-b85a-2713c7cea4ac None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:32.326797 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:32.349342 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:32.403546 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c560fd9b-a9f4-4fea-a825-40b2341e43ba None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:32.436375 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d0b7be-62f2-48e7-b85a-2713c7cea4ac None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:32.444982 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7ea566-fdc5-41c3-a1ca-74a901c3cfbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:32.446253 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7ea566-fdc5-41c3-a1ca-74a901c3cfbf None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:32.446748 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7ea566-fdc5-41c3-a1ca-74a901c3cfbf None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:32.447230 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7ea566-fdc5-41c3-a1ca-74a901c3cfbf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:32.460218 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7ea566-fdc5-41c3-a1ca-74a901c3cfbf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:45:32.470775 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-2a7ea566-fdc5-41c3-a1ca-74a901c3cfbf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 10 12:45:32.470775 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:32.470775 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:32.470775 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:32.470775 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:32.470775 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:32.470775 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:32.470775 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:32.470775 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:32.470775 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:32.470775 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:32.470775 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:32.470775 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:45:32.470775 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:45:32.470775 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:45:32.470775 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:45:32.471742 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:45:32.471742 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:45:32.471742 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:45:32.471742 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:45:32.471742 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:32.471742 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:32.471742 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:32.471742 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:32.471742 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:32.471742 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:32.471742 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:32.471742 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:32.471742 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:32.471742 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:32.471742 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:32.471742 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:32.472645 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:32.472645 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:32.472645 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:32.472645 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:32.472645 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:45:32.472645 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:45:32.472645 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:45:32.472645 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:45:32.472645 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:45:32.472645 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:45:32.472645 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 10 12:45:32.472645 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:32.473287 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 423/847] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Jun 10 12:45:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:32.479499 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6a8bfc-9ed9-4fc3-b7ca-151fc297fd32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:32.479717 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7ea566-fdc5-41c3-a1ca-74a901c3cfbf None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:32.480564 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6a8bfc-9ed9-4fc3-b7ca-151fc297fd32 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:32.480819 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6a8bfc-9ed9-4fc3-b7ca-151fc297fd32 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:32.481101 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6a8bfc-9ed9-4fc3-b7ca-151fc297fd32 None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:32.481955 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6a8bfc-9ed9-4fc3-b7ca-151fc297fd32 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:32.482270 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6a8bfc-9ed9-4fc3-b7ca-151fc297fd32 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:32.484091 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6a8bfc-9ed9-4fc3-b7ca-151fc297fd32 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:32.504908 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 425/848] 10.222.0.27 () {56 vars in 1149 bytes} [Fri Jun 10 12:45:32 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:32.513184 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6a8bfc-9ed9-4fc3-b7ca-151fc297fd32 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:32.594565 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7ea566-fdc5-41c3-a1ca-74a901c3cfbf None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:32.596404 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:32.630686 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c52e82f-1496-40ea-b7cd-958cc1efb8c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:32.631549 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c52e82f-1496-40ea-b7cd-958cc1efb8c3 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:32.631964 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c52e82f-1496-40ea-b7cd-958cc1efb8c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:32.632345 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c52e82f-1496-40ea-b7cd-958cc1efb8c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:32.633202 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c52e82f-1496-40ea-b7cd-958cc1efb8c3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:32.633615 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c52e82f-1496-40ea-b7cd-958cc1efb8c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:32.635076 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c52e82f-1496-40ea-b7cd-958cc1efb8c3 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:32.650015 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 424/849] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Jun 10 12:45:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 12:45:32.651673 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:32.651980 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c52e82f-1496-40ea-b7cd-958cc1efb8c3 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:32.661887 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6a8bfc-9ed9-4fc3-b7ca-151fc297fd32 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:32.701108 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70426d5-cca2-4a43-94bf-fb94831b139e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:32.702151 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70426d5-cca2-4a43-94bf-fb94831b139e None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:32.702692 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70426d5-cca2-4a43-94bf-fb94831b139e None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:32.703173 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70426d5-cca2-4a43-94bf-fb94831b139e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:32.778009 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70426d5-cca2-4a43-94bf-fb94831b139e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:32.780204 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70426d5-cca2-4a43-94bf-fb94831b139e None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:32.782478 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c52e82f-1496-40ea-b7cd-958cc1efb8c3 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:32.827522 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efbefff-0a0a-4bc7-ae51-cc998b1e1b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:32.829155 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efbefff-0a0a-4bc7-ae51-cc998b1e1b18 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:32.829668 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efbefff-0a0a-4bc7-ae51-cc998b1e1b18 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:32.830156 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efbefff-0a0a-4bc7-ae51-cc998b1e1b18 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:32.841761 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efbefff-0a0a-4bc7-ae51-cc998b1e1b18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:32.843448 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efbefff-0a0a-4bc7-ae51-cc998b1e1b18 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:32.862257 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-8efbefff-0a0a-4bc7-ae51-cc998b1e1b18 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:32.862257 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:32.862257 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:32.862257 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:32.862257 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:32.862257 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:32.862257 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:32.862257 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:32.862257 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:32.862257 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:32.862257 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:32.862257 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:32.862257 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 10 12:45:32.862257 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 10 12:45:32.862257 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 10 12:45:32.863203 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 10 12:45:32.863203 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:32.863203 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:32.863203 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:32.863203 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:32.863203 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:32.863203 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:32.863203 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:32.863203 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:32.863203 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:32.863203 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:32.863203 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:32.863203 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:32.863203 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:32.863203 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:32.863203 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:32.863203 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:32.864140 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 10 12:45:32.864140 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 10 12:45:32.864140 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:32.864140 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:32.866402 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 425/850] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:45:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:32.874792 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efbefff-0a0a-4bc7-ae51-cc998b1e1b18 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:32.893333 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:32.898876 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:32.904155 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 426/851] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Jun 10 12:45:32 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 10 12:45:32.913574 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70426d5-cca2-4a43-94bf-fb94831b139e None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:32.991063 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efbefff-0a0a-4bc7-ae51-cc998b1e1b18 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:33.030457 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0b2806-6f45-4b42-8790-4d41de313d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:33.031555 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0b2806-6f45-4b42-8790-4d41de313d20 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:33.032105 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0b2806-6f45-4b42-8790-4d41de313d20 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:33.032481 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0b2806-6f45-4b42-8790-4d41de313d20 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:33.033318 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0b2806-6f45-4b42-8790-4d41de313d20 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:33.033710 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0b2806-6f45-4b42-8790-4d41de313d20 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:33.035183 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0b2806-6f45-4b42-8790-4d41de313d20 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:33.041665 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70426d5-cca2-4a43-94bf-fb94831b139e None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:33.047820 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 426/852] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Jun 10 12:45:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:33.053643 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0b2806-6f45-4b42-8790-4d41de313d20 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:33.087339 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b8625e-28eb-4aa1-a7c9-192475cc7672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:33.088163 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b8625e-28eb-4aa1-a7c9-192475cc7672 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:33.088546 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b8625e-28eb-4aa1-a7c9-192475cc7672 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:33.088934 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b8625e-28eb-4aa1-a7c9-192475cc7672 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:33.100227 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b8625e-28eb-4aa1-a7c9-192475cc7672 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:33.102086 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b8625e-28eb-4aa1-a7c9-192475cc7672 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:33.120288 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-50b8625e-28eb-4aa1-a7c9-192475cc7672 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:33.120288 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:33.120288 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:33.120288 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:33.120288 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:33.120288 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:33.120288 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:33.120288 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:33.120288 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:33.120288 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:33.120288 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:33.120288 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:33.120288 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 10 12:45:33.120288 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 10 12:45:33.120288 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 10 12:45:33.121315 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 10 12:45:33.121315 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:33.121315 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:33.121315 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:33.121315 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:33.121315 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:33.121315 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:33.121315 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:33.121315 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:33.121315 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:33.121315 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:33.121315 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:33.121315 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:33.121315 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:33.121315 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:33.121315 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:33.121315 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:33.122236 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 10 12:45:33.122236 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 10 12:45:33.122236 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:33.122236 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:33.126888 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 427/853] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:45:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:33.127852 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b8625e-28eb-4aa1-a7c9-192475cc7672 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:33.150355 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:33.182043 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:33.185203 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0b2806-6f45-4b42-8790-4d41de313d20 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:33.226388 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8353f4-eb5b-4dbb-9679-c8d6efe1732c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:33.227572 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8353f4-eb5b-4dbb-9679-c8d6efe1732c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:33.228139 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8353f4-eb5b-4dbb-9679-c8d6efe1732c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:33.228636 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8353f4-eb5b-4dbb-9679-c8d6efe1732c None admin] PATH_INFO: `/v3/users/swift` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:33.236498 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8353f4-eb5b-4dbb-9679-c8d6efe1732c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:33.238142 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8353f4-eb5b-4dbb-9679-c8d6efe1732c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:33.240123 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b8625e-28eb-4aa1-a7c9-192475cc7672 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:33.252692 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-5c8353f4-eb5b-4dbb-9679-c8d6efe1732c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 10 12:45:33.252692 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:33.252692 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:33.252692 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:33.252692 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:33.252692 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:33.252692 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:33.252692 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:33.252692 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:33.252692 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:33.252692 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:33.252692 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:33.252692 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 10 12:45:33.252692 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 10 12:45:33.252692 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 10 12:45:33.252692 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 10 12:45:33.253668 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:33.253668 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:33.253668 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 10 12:45:33.253668 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 10 12:45:33.253668 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 10 12:45:33.253668 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 10 12:45:33.253668 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:33.253668 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:33.253668 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:33.253668 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:33.253668 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:33.253668 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:33.253668 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:33.253668 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:33.253668 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:33.253668 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:33.253668 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:33.253668 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:33.254634 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:33.254634 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:33.254634 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 10 12:45:33.254634 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 10 12:45:33.254634 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 10 12:45:33.254634 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 10 12:45:33.254634 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 10 12:45:33.254634 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 10 12:45:33.254634 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 10 12:45:33.254634 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:33.257705 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 427/854] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Jun 10 12:45:33 2022] GET /identity/v3/users/swift => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 10 12:45:33.263478 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8353f4-eb5b-4dbb-9679-c8d6efe1732c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:33.276856 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672052df-a9ba-400f-86fa-4f156d439ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:33.277669 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672052df-a9ba-400f-86fa-4f156d439ead None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:33.277913 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672052df-a9ba-400f-86fa-4f156d439ead None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:33.278198 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672052df-a9ba-400f-86fa-4f156d439ead None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:33.278930 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672052df-a9ba-400f-86fa-4f156d439ead None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:33.279229 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672052df-a9ba-400f-86fa-4f156d439ead None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:33.280752 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672052df-a9ba-400f-86fa-4f156d439ead None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:33.298454 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 428/855] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Jun 10 12:45:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:33.299498 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672052df-a9ba-400f-86fa-4f156d439ead None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:33.371369 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8353f4-eb5b-4dbb-9679-c8d6efe1732c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:33.406898 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672052df-a9ba-400f-86fa-4f156d439ead None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:33.407775 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7d2f7f-1119-4fee-b4c7-550a2bb69ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:33.408817 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7d2f7f-1119-4fee-b4c7-550a2bb69ece None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:33.409375 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7d2f7f-1119-4fee-b4c7-550a2bb69ece None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:33.409835 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7d2f7f-1119-4fee-b4c7-550a2bb69ece None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:33.410850 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7d2f7f-1119-4fee-b4c7-550a2bb69ece None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:33.411601 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7d2f7f-1119-4fee-b4c7-550a2bb69ece None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:33.413276 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7d2f7f-1119-4fee-b4c7-550a2bb69ece None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:33.414332 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:33.417036 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:33.437572 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 428/856] 10.222.0.27 () {56 vars in 1135 bytes} [Fri Jun 10 12:45:33 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:33.438827 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7d2f7f-1119-4fee-b4c7-550a2bb69ece None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:33.446475 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dacaa7c-c71b-476e-bb6c-0e694a640599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:33.447580 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dacaa7c-c71b-476e-bb6c-0e694a640599 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:33.448054 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dacaa7c-c71b-476e-bb6c-0e694a640599 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:33.448541 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dacaa7c-c71b-476e-bb6c-0e694a640599 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:33.458409 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dacaa7c-c71b-476e-bb6c-0e694a640599 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:33.463545 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dacaa7c-c71b-476e-bb6c-0e694a640599 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:33.475685 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-1dacaa7c-c71b-476e-bb6c-0e694a640599 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 10 12:45:33.475685 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:33.475685 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:33.475685 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:33.475685 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:33.475685 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:33.475685 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:33.475685 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:33.475685 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:33.475685 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:33.475685 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:33.475685 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:33.475685 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 10 12:45:33.475685 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 10 12:45:33.475685 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 10 12:45:33.476779 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 10 12:45:33.476779 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:33.476779 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:33.476779 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:33.476779 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:33.476779 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:33.476779 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:33.476779 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:33.476779 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:33.476779 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:33.476779 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:33.476779 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:33.476779 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:33.476779 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:33.476779 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:33.476779 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:33.476779 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:33.477918 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 10 12:45:33.477918 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 10 12:45:33.477918 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 10 12:45:33.477918 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 10 12:45:33.477918 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 10 12:45:33.477918 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 10 12:45:33.477918 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 10 12:45:33.477918 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:33.484757 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 429/857] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Jun 10 12:45:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:33.486888 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dacaa7c-c71b-476e-bb6c-0e694a640599 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:33.550736 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7d2f7f-1119-4fee-b4c7-550a2bb69ece None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:33.588284 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d624d55-f27f-4e0f-8f2e-3ecab11a4d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:33.588944 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d624d55-f27f-4e0f-8f2e-3ecab11a4d51 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:33.589336 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d624d55-f27f-4e0f-8f2e-3ecab11a4d51 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:33.589625 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d624d55-f27f-4e0f-8f2e-3ecab11a4d51 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:33.599901 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dacaa7c-c71b-476e-bb6c-0e694a640599 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:33.637981 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd74506-fd2f-48ad-93bd-33473b35918b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:33.639011 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd74506-fd2f-48ad-93bd-33473b35918b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:33.639554 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd74506-fd2f-48ad-93bd-33473b35918b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:33.640016 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd74506-fd2f-48ad-93bd-33473b35918b None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:33.640955 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd74506-fd2f-48ad-93bd-33473b35918b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:33.641451 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd74506-fd2f-48ad-93bd-33473b35918b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:33.643395 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd74506-fd2f-48ad-93bd-33473b35918b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:33.654952 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:33.662736 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d624d55-f27f-4e0f-8f2e-3ecab11a4d51 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:33.663185 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 430/858] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Jun 10 12:45:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:33.664265 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d624d55-f27f-4e0f-8f2e-3ecab11a4d51 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:33.664814 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd74506-fd2f-48ad-93bd-33473b35918b None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:33.696010 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:33.776364 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd74506-fd2f-48ad-93bd-33473b35918b None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:33.778039 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 429/859] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Jun 10 12:45:33 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 10 12:45:33.779454 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d624d55-f27f-4e0f-8f2e-3ecab11a4d51 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:33.814372 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b132548-9784-4035-90b3-309ffa99e7cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:33.815239 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b132548-9784-4035-90b3-309ffa99e7cf None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:33.815669 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b132548-9784-4035-90b3-309ffa99e7cf None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:33.816079 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b132548-9784-4035-90b3-309ffa99e7cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:33.816980 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b132548-9784-4035-90b3-309ffa99e7cf None admin] RBAC: Adding query filter params (role.id=ea7e8f74523b4322857fb3f4307d171e, scope.project.id=73ea5b23a2d34544ab493f4c5e4034b8, user.id=da66362070694e91bbe2cfbbd161b8a7) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:33.817364 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b132548-9784-4035-90b3-309ffa99e7cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:33.818897 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b132548-9784-4035-90b3-309ffa99e7cf None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:33.825711 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 431/860] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Jun 10 12:45:33 2022] GET /identity/v3/role_assignments?user.id=da66362070694e91bbe2cfbbd161b8a7&scope.project.id=73ea5b23a2d34544ab493f4c5e4034b8&role.id=ea7e8f74523b4322857fb3f4307d171e => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 12:45:33.884407 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:33.888656 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d624d55-f27f-4e0f-8f2e-3ecab11a4d51 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:33.929018 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe15a38a-eaa7-4219-a816-5d6c72be5594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:33.929843 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe15a38a-eaa7-4219-a816-5d6c72be5594 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:33.930130 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe15a38a-eaa7-4219-a816-5d6c72be5594 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:33.930411 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe15a38a-eaa7-4219-a816-5d6c72be5594 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:33.942040 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe15a38a-eaa7-4219-a816-5d6c72be5594 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:33.943516 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe15a38a-eaa7-4219-a816-5d6c72be5594 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:33.962622 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-fe15a38a-eaa7-4219-a816-5d6c72be5594 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:33.962622 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:33.962622 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:33.962622 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:33.962622 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:33.962622 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:33.962622 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:33.962622 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:33.962622 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:33.962622 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:33.962622 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:33.962622 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:33.962622 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 10 12:45:33.962622 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 10 12:45:33.962622 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 10 12:45:33.963577 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 10 12:45:33.963577 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:33.963577 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:33.963577 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:33.963577 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:33.963577 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:33.963577 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:33.963577 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:33.963577 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:33.963577 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:33.963577 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:33.963577 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:33.963577 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:33.963577 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:33.963577 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:33.963577 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:33.963577 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:33.964591 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 10 12:45:33.964591 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 10 12:45:33.964591 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:33.964591 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:33.967984 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 430/861] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:45:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 10 12:45:33.969944 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b132548-9784-4035-90b3-309ffa99e7cf None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:34.056715 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425daa53-c1c9-436c-8844-fbc90dc3ff50 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:34.057001 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425daa53-c1c9-436c-8844-fbc90dc3ff50 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:34.057280 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425daa53-c1c9-436c-8844-fbc90dc3ff50 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:34.058762 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 431/862] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:45:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:45:34.063615 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9605879f-6723-4f66-a4b1-b93c4e05e8e2 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:34.064786 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9605879f-6723-4f66-a4b1-b93c4e05e8e2 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:34.064786 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9605879f-6723-4f66-a4b1-b93c4e05e8e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:34.083241 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-9605879f-6723-4f66-a4b1-b93c4e05e8e2 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:45:34.084266 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:34.085011 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9605879f-6723-4f66-a4b1-b93c4e05e8e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:45:34.097519 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b132548-9784-4035-90b3-309ffa99e7cf None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:34.147711 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2509b950-dd74-423a-9a69-1acf293654b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:34.148506 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2509b950-dd74-423a-9a69-1acf293654b1 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:34.148787 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2509b950-dd74-423a-9a69-1acf293654b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:34.149066 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2509b950-dd74-423a-9a69-1acf293654b1 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:34.149793 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2509b950-dd74-423a-9a69-1acf293654b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:34.150102 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2509b950-dd74-423a-9a69-1acf293654b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:34.151472 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2509b950-dd74-423a-9a69-1acf293654b1 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:34.165292 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 432/863] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Jun 10 12:45:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:34.170919 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2509b950-dd74-423a-9a69-1acf293654b1 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:34.177395 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 432/864] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:45:34 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 10 12:45:34.204098 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9605879f-6723-4f66-a4b1-b93c4e05e8e2 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:34.293475 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:34.296942 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2509b950-dd74-423a-9a69-1acf293654b1 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:34.316971 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9605879f-6723-4f66-a4b1-b93c4e05e8e2 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:34.342786 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28a9914-7879-460c-8642-0fefad6c1f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:34.343608 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28a9914-7879-460c-8642-0fefad6c1f82 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:34.344135 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28a9914-7879-460c-8642-0fefad6c1f82 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:34.344409 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28a9914-7879-460c-8642-0fefad6c1f82 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:34.354420 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28a9914-7879-460c-8642-0fefad6c1f82 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:34.355877 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28a9914-7879-460c-8642-0fefad6c1f82 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:34.356628 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51467870-42f4-48ef-af91-5a87926365d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:34.357653 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51467870-42f4-48ef-af91-5a87926365d5 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:34.358036 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51467870-42f4-48ef-af91-5a87926365d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:34.358502 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51467870-42f4-48ef-af91-5a87926365d5 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:34.359266 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51467870-42f4-48ef-af91-5a87926365d5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:34.360788 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51467870-42f4-48ef-af91-5a87926365d5 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:34.371331 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-e28a9914-7879-460c-8642-0fefad6c1f82 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 10 12:45:34.371331 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:34.371331 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:34.371331 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:34.371331 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:34.371331 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:34.371331 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:34.371331 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:34.371331 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:34.371331 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:34.371331 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:34.371331 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:34.371331 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 10 12:45:34.371331 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 10 12:45:34.371331 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 10 12:45:34.372363 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 10 12:45:34.372363 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:34.372363 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:34.372363 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:34.372363 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:34.372363 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:34.372363 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:34.372363 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:34.372363 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:34.372363 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:34.372363 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:34.372363 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:34.372363 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:34.372363 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:34.372363 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:34.372363 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:34.372363 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:34.373287 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 10 12:45:34.373287 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 10 12:45:34.373287 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 10 12:45:34.373287 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 10 12:45:34.373287 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 10 12:45:34.373287 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 10 12:45:34.373287 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 10 12:45:34.373287 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:34.373287 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 433/865] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Jun 10 12:45:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:34.374223 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-51467870-42f4-48ef-af91-5a87926365d5 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 10 12:45:34.374223 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:34.374223 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 10 12:45:34.374223 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 10 12:45:34.374223 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 10 12:45:34.374223 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 10 12:45:34.374223 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 10 12:45:34.374223 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 10 12:45:34.374223 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 10 12:45:34.374223 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:34.374223 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 10 12:45:34.374223 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:34.374223 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:34.374223 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:34.374223 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:34.374223 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:34.375157 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:34.375157 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:34.375157 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:34.375157 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:34.375157 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:34.375157 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:34.375157 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:34.375157 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 10 12:45:34.375157 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 10 12:45:34.375157 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 10 12:45:34.375157 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 10 12:45:34.375157 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:34.375157 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:34.375157 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:34.375157 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:34.375157 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:34.376029 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:34.376029 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:34.376029 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:34.376029 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:34.376029 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:34.376029 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:34.376029 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:34.376029 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:34.376029 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:34.376029 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:34.376029 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:34.376029 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 10 12:45:34.376029 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 10 12:45:34.376029 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 10 12:45:34.376029 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:34.378267 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28a9914-7879-460c-8642-0fefad6c1f82 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:34.380263 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 433/866] 10.222.0.27 () {56 vars in 1132 bytes} [Fri Jun 10 12:45:34 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:34.386152 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51467870-42f4-48ef-af91-5a87926365d5 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:34.394304 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:34.499622 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51467870-42f4-48ef-af91-5a87926365d5 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:34.515584 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28a9914-7879-460c-8642-0fefad6c1f82 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:34.546335 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1d59b1-9b1a-4619-a2d0-3b6ddc932a75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:34.547758 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1d59b1-9b1a-4619-a2d0-3b6ddc932a75 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:34.548310 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1d59b1-9b1a-4619-a2d0-3b6ddc932a75 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:34.548731 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1d59b1-9b1a-4619-a2d0-3b6ddc932a75 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:34.549858 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1d59b1-9b1a-4619-a2d0-3b6ddc932a75 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:34.550689 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1d59b1-9b1a-4619-a2d0-3b6ddc932a75 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:34.552493 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1d59b1-9b1a-4619-a2d0-3b6ddc932a75 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:34.555446 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac985a6-5cd9-4605-aa27-472de86c31a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:34.556241 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac985a6-5cd9-4605-aa27-472de86c31a0 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:34.556629 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac985a6-5cd9-4605-aa27-472de86c31a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:34.557013 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac985a6-5cd9-4605-aa27-472de86c31a0 None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:34.558053 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac985a6-5cd9-4605-aa27-472de86c31a0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:34.558170 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 434/867] 10.222.0.27 () {56 vars in 1126 bytes} [Fri Jun 10 12:45:34 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:34.558249 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac985a6-5cd9-4605-aa27-472de86c31a0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:34.559130 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d732a282-5fed-4291-b19c-7d2f515c373f None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:34.559417 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d732a282-5fed-4291-b19c-7d2f515c373f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:34.559589 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac985a6-5cd9-4605-aa27-472de86c31a0 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:34.559686 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d732a282-5fed-4291-b19c-7d2f515c373f None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:34.561433 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 435/868] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:45:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:45:34.564954 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d732a282-5fed-4291-b19c-7d2f515c373f None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:34.570494 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:34.573204 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 434/869] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Jun 10 12:45:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:34.574782 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e703fe64-3d5f-4310-b20f-0e6e942bd9a6 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:34.575282 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e703fe64-3d5f-4310-b20f-0e6e942bd9a6 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:34.575766 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e703fe64-3d5f-4310-b20f-0e6e942bd9a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:34.595592 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-e703fe64-3d5f-4310-b20f-0e6e942bd9a6 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:45:34.596823 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e703fe64-3d5f-4310-b20f-0e6e942bd9a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:45:34.665089 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:34.670244 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 435/870] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:45:34 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 10 12:45:34.671052 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e703fe64-3d5f-4310-b20f-0e6e942bd9a6 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:34.677393 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d732a282-5fed-4291-b19c-7d2f515c373f None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:34.721259 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd385633-bae7-4c9d-8561-7aa1a15eeb5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:34.722613 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd385633-bae7-4c9d-8561-7aa1a15eeb5d None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:34.723093 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd385633-bae7-4c9d-8561-7aa1a15eeb5d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:34.723558 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd385633-bae7-4c9d-8561-7aa1a15eeb5d None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:34.724513 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd385633-bae7-4c9d-8561-7aa1a15eeb5d None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:34.725045 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd385633-bae7-4c9d-8561-7aa1a15eeb5d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:34.727207 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd385633-bae7-4c9d-8561-7aa1a15eeb5d None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:34.732213 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 436/871] 10.222.0.27 () {56 vars in 1126 bytes} [Fri Jun 10 12:45:34 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:34.732984 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd385633-bae7-4c9d-8561-7aa1a15eeb5d None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:34.801220 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e703fe64-3d5f-4310-b20f-0e6e942bd9a6 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:34.812742 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:34.850626 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd385633-bae7-4c9d-8561-7aa1a15eeb5d None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:34.858872 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93857a64-ebaf-4e28-ab13-f04b490a989d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:34.859553 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93857a64-ebaf-4e28-ab13-f04b490a989d None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:34.859831 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93857a64-ebaf-4e28-ab13-f04b490a989d None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:34.860132 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93857a64-ebaf-4e28-ab13-f04b490a989d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:34.860978 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93857a64-ebaf-4e28-ab13-f04b490a989d None admin] RBAC: Adding query filter params (role.id=b43d5a8247e244869c2c70f986f42e8a, scope.project.id=73ea5b23a2d34544ab493f4c5e4034b8, user.id=fb63b9cd74bf438aa25b72cef8337fec) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:34.861288 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93857a64-ebaf-4e28-ab13-f04b490a989d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:34.864828 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93857a64-ebaf-4e28-ab13-f04b490a989d None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:34.880005 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 436/872] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Jun 10 12:45:34 2022] GET /identity/v3/role_assignments?user.id=fb63b9cd74bf438aa25b72cef8337fec&scope.project.id=73ea5b23a2d34544ab493f4c5e4034b8&role.id=b43d5a8247e244869c2c70f986f42e8a => generated 608 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:34.907731 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53888d02-d680-41e8-bdc2-024154b20cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:34.908452 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53888d02-d680-41e8-bdc2-024154b20cef None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:34.908739 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53888d02-d680-41e8-bdc2-024154b20cef None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:34.909018 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53888d02-d680-41e8-bdc2-024154b20cef None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:34.909769 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53888d02-d680-41e8-bdc2-024154b20cef None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:34.911466 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53888d02-d680-41e8-bdc2-024154b20cef None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:34.975340 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 437/873] 10.222.0.27 () {60 vars in 1141 bytes} [Fri Jun 10 12:45:34 2022] POST /identity/v3/services => generated 251 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 12:45:36.154971 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0368908-35f0-44c3-be39-fbdb1272100a None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:36.155451 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0368908-35f0-44c3-be39-fbdb1272100a None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:36.155451 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0368908-35f0-44c3-be39-fbdb1272100a None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:36.157055 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 437/874] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:45:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:45:36.162582 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba240557-b8ba-4db0-a659-e36df502cdce None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:36.162971 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba240557-b8ba-4db0-a659-e36df502cdce None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:36.163143 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba240557-b8ba-4db0-a659-e36df502cdce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:36.179972 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-ba240557-b8ba-4db0-a659-e36df502cdce None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:45:36.181148 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ba240557-b8ba-4db0-a659-e36df502cdce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:45:36.258416 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 438/875] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:45:36 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 10 12:45:36.281571 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0368908-35f0-44c3-be39-fbdb1272100a None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:36.348334 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:36.392676 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0368908-35f0-44c3-be39-fbdb1272100a None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:36.429219 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf45b3e0-db93-4a61-a707-d8ae3fd7a312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:36.429961 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf45b3e0-db93-4a61-a707-d8ae3fd7a312 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:36.430306 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf45b3e0-db93-4a61-a707-d8ae3fd7a312 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:36.430587 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf45b3e0-db93-4a61-a707-d8ae3fd7a312 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:36.441215 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf45b3e0-db93-4a61-a707-d8ae3fd7a312 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:45:36.450132 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-cf45b3e0-db93-4a61-a707-d8ae3fd7a312 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 10 12:45:36.450132 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:36.450132 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:36.450132 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:36.450132 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:36.450132 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:36.450132 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:36.450132 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:36.450132 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:36.450132 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:36.450132 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:36.450132 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:36.450132 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:45:36.450132 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:45:36.450132 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:45:36.450132 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:45:36.451064 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:45:36.451064 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:45:36.451064 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:45:36.451064 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:45:36.451064 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:36.451064 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:36.451064 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:36.451064 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:36.451064 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:36.451064 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:36.451064 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:36.451064 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:36.451064 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:36.451064 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:36.451064 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:36.451064 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:36.451953 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:36.451953 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:36.451953 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:36.451953 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:36.451953 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:45:36.451953 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:45:36.451953 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:45:36.451953 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:45:36.451953 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:45:36.451953 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:45:36.451953 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 10 12:45:36.451953 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:36.451953 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 438/876] 10.222.0.27 () {56 vars in 1120 bytes} [Fri Jun 10 12:45:36 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:36.460940 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba240557-b8ba-4db0-a659-e36df502cdce None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:36.517649 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:36.562997 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba240557-b8ba-4db0-a659-e36df502cdce None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:36.599525 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47877ab-3c45-4fd2-9c20-12f062869ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:36.600254 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47877ab-3c45-4fd2-9c20-12f062869ae0 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:36.600543 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47877ab-3c45-4fd2-9c20-12f062869ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:36.600823 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47877ab-3c45-4fd2-9c20-12f062869ae0 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:36.601580 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47877ab-3c45-4fd2-9c20-12f062869ae0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:36.601887 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47877ab-3c45-4fd2-9c20-12f062869ae0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:36.603259 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47877ab-3c45-4fd2-9c20-12f062869ae0 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:36.612527 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 439/877] 10.222.0.27 () {56 vars in 1115 bytes} [Fri Jun 10 12:45:36 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:36.618579 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf45b3e0-db93-4a61-a707-d8ae3fd7a312 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:36.728280 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf45b3e0-db93-4a61-a707-d8ae3fd7a312 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:36.752636 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:36.763087 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4828571-8106-4976-83b3-f9df79c919da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:36.763894 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4828571-8106-4976-83b3-f9df79c919da None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:36.764227 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4828571-8106-4976-83b3-f9df79c919da None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:36.764506 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4828571-8106-4976-83b3-f9df79c919da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:36.775155 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4828571-8106-4976-83b3-f9df79c919da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:36.776608 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4828571-8106-4976-83b3-f9df79c919da None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:36.799243 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-d4828571-8106-4976-83b3-f9df79c919da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:36.799243 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:36.799243 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:36.799243 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:36.799243 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:36.799243 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:36.799243 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:36.799243 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:36.799243 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:36.799243 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:36.799243 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:36.799243 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:36.799243 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 10 12:45:36.799243 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 10 12:45:36.799243 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 10 12:45:36.800264 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 10 12:45:36.800264 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:36.800264 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:36.800264 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:36.800264 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:36.800264 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:36.800264 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:36.800264 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:36.800264 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:36.800264 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:36.800264 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:36.800264 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:36.800264 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:36.800264 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:36.800264 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:36.800264 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:36.800264 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:36.801196 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 10 12:45:36.801196 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 10 12:45:36.801196 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:36.801196 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:36.801196 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 439/878] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:45:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:36.808179 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47877ab-3c45-4fd2-9c20-12f062869ae0 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:36.915826 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47877ab-3c45-4fd2-9c20-12f062869ae0 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:36.933376 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:36.954017 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fc29d9-a764-4b74-8e38-5ea7ee665643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:36.954736 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fc29d9-a764-4b74-8e38-5ea7ee665643 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:36.955033 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fc29d9-a764-4b74-8e38-5ea7ee665643 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:36.955317 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fc29d9-a764-4b74-8e38-5ea7ee665643 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:36.956115 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fc29d9-a764-4b74-8e38-5ea7ee665643 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:36.956432 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fc29d9-a764-4b74-8e38-5ea7ee665643 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:36.957997 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fc29d9-a764-4b74-8e38-5ea7ee665643 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:36.972964 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 440/879] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Jun 10 12:45:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:36.979358 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4828571-8106-4976-83b3-f9df79c919da None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:37.020054 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcc86f1-6882-4e51-9cff-a48d8ab880f2 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:37.020377 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcc86f1-6882-4e51-9cff-a48d8ab880f2 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:37.020620 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcc86f1-6882-4e51-9cff-a48d8ab880f2 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:37.022392 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 441/880] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:45:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 10 12:45:37.027730 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9496f7cf-01d7-45b1-bf9c-65b77fb7f961 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:37.028062 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9496f7cf-01d7-45b1-bf9c-65b77fb7f961 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:37.028344 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9496f7cf-01d7-45b1-bf9c-65b77fb7f961 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:37.029731 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 442/881] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:45:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 10 12:45:37.030622 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8627f002-5898-424f-9a8b-4c32ab89b927 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:37.030900 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8627f002-5898-424f-9a8b-4c32ab89b927 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:37.031184 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8627f002-5898-424f-9a8b-4c32ab89b927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:37.051825 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-8627f002-5898-424f-9a8b-4c32ab89b927 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:45:37.053901 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8627f002-5898-424f-9a8b-4c32ab89b927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:45:37.086799 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4828571-8106-4976-83b3-f9df79c919da None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:37.122195 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d87d08-c0be-4541-bdb0-5ff4e7d490ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:37.123006 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d87d08-c0be-4541-bdb0-5ff4e7d490ee None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:37.123287 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d87d08-c0be-4541-bdb0-5ff4e7d490ee None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:37.123553 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d87d08-c0be-4541-bdb0-5ff4e7d490ee None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:37.131180 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d87d08-c0be-4541-bdb0-5ff4e7d490ee None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:37.132135 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 443/882] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:45:37 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 10 12:45:37.132657 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d87d08-c0be-4541-bdb0-5ff4e7d490ee None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:37.137278 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277a957b-d11f-499a-9939-359233d15f9d None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:37.142483 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277a957b-d11f-499a-9939-359233d15f9d None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:37.142483 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277a957b-d11f-499a-9939-359233d15f9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:37.148099 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-f1d87d08-c0be-4541-bdb0-5ff4e7d490ee None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 10 12:45:37.148099 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:37.148099 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:37.148099 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:37.148099 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:37.148099 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:37.148099 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:37.148099 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:37.148099 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:37.148099 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:37.148099 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:37.148099 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:37.148099 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 10 12:45:37.148099 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 10 12:45:37.148099 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 10 12:45:37.148099 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 10 12:45:37.149133 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:37.149133 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:37.149133 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 10 12:45:37.149133 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 10 12:45:37.149133 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 10 12:45:37.149133 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 10 12:45:37.149133 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:37.149133 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:37.149133 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:37.149133 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:37.149133 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:37.149133 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:37.149133 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:37.149133 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:37.149133 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:37.149133 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:37.149133 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:37.149133 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:37.150133 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:37.150133 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:37.150133 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 10 12:45:37.150133 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 10 12:45:37.150133 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 10 12:45:37.150133 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 10 12:45:37.150133 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 10 12:45:37.150133 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 10 12:45:37.150133 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 10 12:45:37.150133 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:37.154225 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 440/883] 10.222.0.27 () {56 vars in 1117 bytes} [Fri Jun 10 12:45:36 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:37.155220 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d87d08-c0be-4541-bdb0-5ff4e7d490ee None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:37.163503 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-277a957b-d11f-499a-9939-359233d15f9d None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:45:37.164786 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-277a957b-d11f-499a-9939-359233d15f9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:45:37.197060 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:37.243577 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 444/884] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:45:37 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 10 12:45:37.244334 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277a957b-d11f-499a-9939-359233d15f9d None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:37.262107 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d87d08-c0be-4541-bdb0-5ff4e7d490ee None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:37.285585 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:37.298136 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea169e97-7e5b-4bde-bc9f-5e71f5ab28ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:37.298806 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea169e97-7e5b-4bde-bc9f-5e71f5ab28ed None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:37.299091 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea169e97-7e5b-4bde-bc9f-5e71f5ab28ed None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:37.299361 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea169e97-7e5b-4bde-bc9f-5e71f5ab28ed None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:37.300236 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea169e97-7e5b-4bde-bc9f-5e71f5ab28ed None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:37.300549 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea169e97-7e5b-4bde-bc9f-5e71f5ab28ed None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:37.301985 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea169e97-7e5b-4bde-bc9f-5e71f5ab28ed None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:37.318711 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 441/885] 10.222.0.27 () {56 vars in 1149 bytes} [Fri Jun 10 12:45:37 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:37.319983 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543a0837-1bc0-4f61-9e76-d76ff2435f80 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:37.320275 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543a0837-1bc0-4f61-9e76-d76ff2435f80 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:37.320556 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543a0837-1bc0-4f61-9e76-d76ff2435f80 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:37.321776 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 442/886] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:45:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:45:37.322465 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543a0837-1bc0-4f61-9e76-d76ff2435f80 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:37.350632 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277a957b-d11f-499a-9939-359233d15f9d None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:37.383298 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7bd994-1b67-4a13-88c0-650b97ba7926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:37.384007 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7bd994-1b67-4a13-88c0-650b97ba7926 None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:37.384292 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7bd994-1b67-4a13-88c0-650b97ba7926 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:37.384561 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7bd994-1b67-4a13-88c0-650b97ba7926 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:37.385271 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7bd994-1b67-4a13-88c0-650b97ba7926 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:37.386610 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7bd994-1b67-4a13-88c0-650b97ba7926 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:37.415939 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:37.423221 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543a0837-1bc0-4f61-9e76-d76ff2435f80 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:37.446311 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 445/887] 10.222.0.27 () {60 vars in 1141 bytes} [Fri Jun 10 12:45:37 2022] POST /identity/v3/services => generated 269 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 12:45:37.447158 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7bd994-1b67-4a13-88c0-650b97ba7926 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:37.464894 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2580b96-0ea0-4779-aaf4-1e0937f40084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:37.465987 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2580b96-0ea0-4779-aaf4-1e0937f40084 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:37.466457 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2580b96-0ea0-4779-aaf4-1e0937f40084 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:37.466884 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2580b96-0ea0-4779-aaf4-1e0937f40084 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:37.467711 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2580b96-0ea0-4779-aaf4-1e0937f40084 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:37.469347 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2580b96-0ea0-4779-aaf4-1e0937f40084 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:37.482923 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-d2580b96-0ea0-4779-aaf4-1e0937f40084 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 10 12:45:37.482923 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:37.482923 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 10 12:45:37.482923 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 10 12:45:37.482923 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 10 12:45:37.482923 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 10 12:45:37.482923 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 10 12:45:37.482923 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 10 12:45:37.482923 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 10 12:45:37.482923 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:37.482923 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 10 12:45:37.482923 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:37.482923 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:37.482923 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:37.482923 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:37.482923 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:37.482923 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:37.484099 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:37.484099 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:37.484099 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:37.484099 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:37.484099 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:37.484099 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:37.484099 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 10 12:45:37.484099 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 10 12:45:37.484099 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 10 12:45:37.484099 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 10 12:45:37.484099 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:37.484099 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:37.484099 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:37.484099 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:37.484099 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:37.484099 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:37.484999 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:37.484999 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:37.484999 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:37.484999 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:37.484999 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:37.484999 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:37.484999 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:37.484999 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:37.484999 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:37.484999 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:37.484999 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 10 12:45:37.484999 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 10 12:45:37.484999 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 10 12:45:37.484999 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:37.484999 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 443/888] 10.222.0.27 () {56 vars in 1126 bytes} [Fri Jun 10 12:45:37 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:37.485845 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ff3ff6-70ea-469d-aa5c-d8db7653ed02 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:37.485845 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ff3ff6-70ea-469d-aa5c-d8db7653ed02 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:37.486806 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ff3ff6-70ea-469d-aa5c-d8db7653ed02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:37.504810 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-b2ff3ff6-70ea-469d-aa5c-d8db7653ed02 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:45:37.505959 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b2ff3ff6-70ea-469d-aa5c-d8db7653ed02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:45:37.561764 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:37.568278 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7bd994-1b67-4a13-88c0-650b97ba7926 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:37.588614 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 444/889] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:45:37 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 10 12:45:37.589844 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ff3ff6-70ea-469d-aa5c-d8db7653ed02 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:37.623233 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcbc758-43af-4498-9415-bd995ce8ac1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:37.623921 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcbc758-43af-4498-9415-bd995ce8ac1a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:37.624206 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcbc758-43af-4498-9415-bd995ce8ac1a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:37.624925 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcbc758-43af-4498-9415-bd995ce8ac1a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:37.694490 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:37.700263 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcbc758-43af-4498-9415-bd995ce8ac1a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:37.701769 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcbc758-43af-4498-9415-bd995ce8ac1a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:37.712978 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ff3ff6-70ea-469d-aa5c-d8db7653ed02 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:37.750871 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2f9916-c996-4fc9-9838-5e6f04bc8195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:37.751622 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2f9916-c996-4fc9-9838-5e6f04bc8195 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:37.752003 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2f9916-c996-4fc9-9838-5e6f04bc8195 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:37.752344 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2f9916-c996-4fc9-9838-5e6f04bc8195 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:37.753120 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2f9916-c996-4fc9-9838-5e6f04bc8195 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:37.753497 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2f9916-c996-4fc9-9838-5e6f04bc8195 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:37.754988 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2f9916-c996-4fc9-9838-5e6f04bc8195 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:37.760379 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 445/890] 10.222.0.27 () {56 vars in 1122 bytes} [Fri Jun 10 12:45:37 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:37.761203 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2f9916-c996-4fc9-9838-5e6f04bc8195 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:37.804273 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:37.815909 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 446/891] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Jun 10 12:45:37 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 10 12:45:37.815909 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcbc758-43af-4498-9415-bd995ce8ac1a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:37.879351 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2f9916-c996-4fc9-9838-5e6f04bc8195 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:37.915264 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8956b593-f6a8-4b0f-97a7-cb0efeb74809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:37.916211 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8956b593-f6a8-4b0f-97a7-cb0efeb74809 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:37.916558 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8956b593-f6a8-4b0f-97a7-cb0efeb74809 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:37.916903 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8956b593-f6a8-4b0f-97a7-cb0efeb74809 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:37.917635 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8956b593-f6a8-4b0f-97a7-cb0efeb74809 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:37.919060 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8956b593-f6a8-4b0f-97a7-cb0efeb74809 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:37.931335 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-8956b593-f6a8-4b0f-97a7-cb0efeb74809 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 10 12:45:37.931335 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:37.931335 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 10 12:45:37.931335 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 10 12:45:37.931335 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 10 12:45:37.931335 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 10 12:45:37.931335 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 10 12:45:37.931335 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 10 12:45:37.931335 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 10 12:45:37.931335 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:37.931335 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 10 12:45:37.931335 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:37.931335 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:37.931335 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:37.931335 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:37.931335 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:37.931335 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:37.932659 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:37.932659 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:37.932659 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:37.932659 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:37.932659 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:37.932659 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:37.932659 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 10 12:45:37.932659 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 10 12:45:37.932659 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 10 12:45:37.932659 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 10 12:45:37.932659 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:37.932659 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:37.932659 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:37.932659 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:37.932659 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:37.932659 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:37.933532 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:37.933532 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:37.933532 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:37.933532 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:37.933532 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:37.933532 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:37.933532 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:37.933532 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:37.933532 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:37.933532 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:37.933532 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 10 12:45:37.933532 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 10 12:45:37.933532 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 10 12:45:37.933532 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:37.934281 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcbc758-43af-4498-9415-bd995ce8ac1a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:37.934357 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 446/892] 10.222.0.27 () {56 vars in 1114 bytes} [Fri Jun 10 12:45:37 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:37.934357 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8956b593-f6a8-4b0f-97a7-cb0efeb74809 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:37.953626 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:37.976134 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b7c96e-67f3-4bc9-814a-2a6ecf4c35c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:37.977125 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b7c96e-67f3-4bc9-814a-2a6ecf4c35c2 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:37.977623 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b7c96e-67f3-4bc9-814a-2a6ecf4c35c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:37.978494 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b7c96e-67f3-4bc9-814a-2a6ecf4c35c2 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:37.979441 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b7c96e-67f3-4bc9-814a-2a6ecf4c35c2 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:37.979969 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b7c96e-67f3-4bc9-814a-2a6ecf4c35c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:37.982299 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b7c96e-67f3-4bc9-814a-2a6ecf4c35c2 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:37.992033 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 447/893] 10.222.0.27 () {56 vars in 1122 bytes} [Fri Jun 10 12:45:37 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:37.992917 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b7c96e-67f3-4bc9-814a-2a6ecf4c35c2 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:38.059806 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8956b593-f6a8-4b0f-97a7-cb0efeb74809 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:38.061058 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:38.107014 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b7c96e-67f3-4bc9-814a-2a6ecf4c35c2 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:38.108447 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d65ba01-3424-455a-b07f-8317dbc5ce75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:38.109426 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d65ba01-3424-455a-b07f-8317dbc5ce75 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:38.109592 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d65ba01-3424-455a-b07f-8317dbc5ce75 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:38.109864 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d65ba01-3424-455a-b07f-8317dbc5ce75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:38.121660 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d65ba01-3424-455a-b07f-8317dbc5ce75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:38.123206 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d65ba01-3424-455a-b07f-8317dbc5ce75 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:38.143644 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-0d65ba01-3424-455a-b07f-8317dbc5ce75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:38.143644 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:38.143644 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:38.143644 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:38.143644 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:38.143644 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:38.143644 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:38.143644 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:38.143644 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:38.143644 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:38.143644 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:38.143644 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:38.143644 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 10 12:45:38.143644 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 10 12:45:38.143644 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 10 12:45:38.144696 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 10 12:45:38.144696 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:38.144696 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:38.144696 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:38.144696 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:38.144696 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:38.144696 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:38.144696 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:38.144696 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:38.144696 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:38.144696 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:38.144696 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:38.144696 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:38.144696 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:38.144696 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:38.144696 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:38.144696 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:38.145611 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 10 12:45:38.145611 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 10 12:45:38.145611 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:38.145611 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:38.148012 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 447/894] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:45:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:38.150907 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d65ba01-3424-455a-b07f-8317dbc5ce75 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:38.154242 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a00093-11b9-420b-b1f1-8ad41c1da694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:38.154928 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a00093-11b9-420b-b1f1-8ad41c1da694 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:38.155216 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a00093-11b9-420b-b1f1-8ad41c1da694 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:38.155497 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a00093-11b9-420b-b1f1-8ad41c1da694 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:38.156282 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a00093-11b9-420b-b1f1-8ad41c1da694 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:38.156609 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a00093-11b9-420b-b1f1-8ad41c1da694 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:38.157977 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a00093-11b9-420b-b1f1-8ad41c1da694 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:38.167957 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 448/895] 10.222.0.27 () {56 vars in 1114 bytes} [Fri Jun 10 12:45:37 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:38.171199 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a00093-11b9-420b-b1f1-8ad41c1da694 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:38.252285 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:38.276809 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d65ba01-3424-455a-b07f-8317dbc5ce75 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:38.285325 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a00093-11b9-420b-b1f1-8ad41c1da694 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:38.313016 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:38.323679 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1528f5b-f6d9-4daf-967e-f41f50d82d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:38.324389 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1528f5b-f6d9-4daf-967e-f41f50d82d25 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:38.324672 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1528f5b-f6d9-4daf-967e-f41f50d82d25 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:38.324952 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1528f5b-f6d9-4daf-967e-f41f50d82d25 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:38.325660 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1528f5b-f6d9-4daf-967e-f41f50d82d25 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:38.329944 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1528f5b-f6d9-4daf-967e-f41f50d82d25 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:38.330050 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b60c93f-f97a-4de5-83dc-9af54c33f151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:38.330050 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b60c93f-f97a-4de5-83dc-9af54c33f151 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:38.330050 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b60c93f-f97a-4de5-83dc-9af54c33f151 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:38.330366 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1528f5b-f6d9-4daf-967e-f41f50d82d25 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:38.330446 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b60c93f-f97a-4de5-83dc-9af54c33f151 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:38.330446 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b60c93f-f97a-4de5-83dc-9af54c33f151 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:38.330446 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b60c93f-f97a-4de5-83dc-9af54c33f151 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:38.331693 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b60c93f-f97a-4de5-83dc-9af54c33f151 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:38.337876 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 449/896] 10.222.0.27 () {56 vars in 1114 bytes} [Fri Jun 10 12:45:38 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 12:45:38.338822 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1528f5b-f6d9-4daf-967e-f41f50d82d25 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:38.355246 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 448/897] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Jun 10 12:45:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:38.355781 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b60c93f-f97a-4de5-83dc-9af54c33f151 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:38.456279 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1528f5b-f6d9-4daf-967e-f41f50d82d25 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:38.493644 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5197bea6-c861-40bc-8ab9-906a1aee6d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:38.494829 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5197bea6-c861-40bc-8ab9-906a1aee6d1b None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:38.495263 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5197bea6-c861-40bc-8ab9-906a1aee6d1b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:38.495685 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5197bea6-c861-40bc-8ab9-906a1aee6d1b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:38.496417 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b60c93f-f97a-4de5-83dc-9af54c33f151 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:38.496839 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5197bea6-c861-40bc-8ab9-906a1aee6d1b None admin] RBAC: Adding query filter params (interface=public, service_id=8582776a58bb4377a7ed5c8177498b4d, region_id=RegionOne) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:38.497298 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5197bea6-c861-40bc-8ab9-906a1aee6d1b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:38.499974 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5197bea6-c861-40bc-8ab9-906a1aee6d1b None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:38.510944 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 450/898] 10.222.0.27 () {56 vars in 1251 bytes} [Fri Jun 10 12:45:38 2022] GET /identity/v3/endpoints?service_id=8582776a58bb4377a7ed5c8177498b4d&interface=public®ion_id=RegionOne => generated 194 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:38.513446 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5197bea6-c861-40bc-8ab9-906a1aee6d1b None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:38.527108 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:38.539041 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d23051b-ae25-4d18-8693-b21192af353f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:38.539843 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d23051b-ae25-4d18-8693-b21192af353f None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:38.540169 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d23051b-ae25-4d18-8693-b21192af353f None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:38.540447 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d23051b-ae25-4d18-8693-b21192af353f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:38.547556 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d23051b-ae25-4d18-8693-b21192af353f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:38.548890 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d23051b-ae25-4d18-8693-b21192af353f None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:38.569281 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-4d23051b-ae25-4d18-8693-b21192af353f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 10 12:45:38.569281 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:38.569281 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:38.569281 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:38.569281 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:38.569281 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:38.569281 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:38.569281 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:38.569281 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:38.569281 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:38.569281 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:38.569281 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:38.569281 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 10 12:45:38.569281 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 10 12:45:38.569281 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 10 12:45:38.570494 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 10 12:45:38.570494 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:38.570494 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:38.570494 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:38.570494 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:38.570494 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:38.570494 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:38.570494 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:38.570494 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:38.570494 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:38.570494 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:38.570494 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:38.570494 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:38.570494 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:38.570494 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:38.570494 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:38.570494 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:38.571404 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 10 12:45:38.571404 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 10 12:45:38.571404 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 10 12:45:38.571404 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 10 12:45:38.571404 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 10 12:45:38.571404 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 10 12:45:38.571404 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 10 12:45:38.571404 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:38.575258 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 449/899] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Jun 10 12:45:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 10 12:45:38.576118 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d23051b-ae25-4d18-8693-b21192af353f None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:38.592193 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:38.655936 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5197bea6-c861-40bc-8ab9-906a1aee6d1b None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:38.696668 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d23051b-ae25-4d18-8693-b21192af353f None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:38.701302 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e055ac-baa4-47ba-bf94-ea201492a5c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:38.702247 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e055ac-baa4-47ba-bf94-ea201492a5c7 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:38.702713 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e055ac-baa4-47ba-bf94-ea201492a5c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:38.703167 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e055ac-baa4-47ba-bf94-ea201492a5c7 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:38.704070 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e055ac-baa4-47ba-bf94-ea201492a5c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:38.706078 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e055ac-baa4-47ba-bf94-ea201492a5c7 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:38.719892 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 451/900] 10.222.0.27 () {56 vars in 1087 bytes} [Fri Jun 10 12:45:38 2022] GET /identity/v3/services => generated 1268 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 12:45:38.740012 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bf450d-d97d-4061-b6a8-8f7649a4f286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:38.740777 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bf450d-d97d-4061-b6a8-8f7649a4f286 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:38.741058 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bf450d-d97d-4061-b6a8-8f7649a4f286 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:38.741330 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bf450d-d97d-4061-b6a8-8f7649a4f286 None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:38.742140 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bf450d-d97d-4061-b6a8-8f7649a4f286 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:38.742447 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bf450d-d97d-4061-b6a8-8f7649a4f286 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:38.743935 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bf450d-d97d-4061-b6a8-8f7649a4f286 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:38.757623 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 450/901] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Jun 10 12:45:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:38.765091 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e055ac-baa4-47ba-bf94-ea201492a5c7 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:38.878892 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e055ac-baa4-47ba-bf94-ea201492a5c7 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:38.899315 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:38.916959 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ad38ba-890d-4c80-bcd2-952e4bd7e5a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:38.918949 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ad38ba-890d-4c80-bcd2-952e4bd7e5a0 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:38.918949 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ad38ba-890d-4c80-bcd2-952e4bd7e5a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:38.918949 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ad38ba-890d-4c80-bcd2-952e4bd7e5a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:38.919159 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ad38ba-890d-4c80-bcd2-952e4bd7e5a0 None admin] RBAC: Adding query filter params (role.id=55ae4712a75e4f9bb542575264d55f64, scope.project.id=73ea5b23a2d34544ab493f4c5e4034b8, user.id=da66362070694e91bbe2cfbbd161b8a7) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:38.919405 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ad38ba-890d-4c80-bcd2-952e4bd7e5a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:38.920788 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ad38ba-890d-4c80-bcd2-952e4bd7e5a0 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:38.926537 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 452/902] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Jun 10 12:45:38 2022] GET /identity/v3/role_assignments?user.id=da66362070694e91bbe2cfbbd161b8a7&scope.project.id=73ea5b23a2d34544ab493f4c5e4034b8&role.id=55ae4712a75e4f9bb542575264d55f64 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:39.622748 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce80906e-32f4-4d28-b089-d553a62d5eee None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:39.623888 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce80906e-32f4-4d28-b089-d553a62d5eee None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:39.624394 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce80906e-32f4-4d28-b089-d553a62d5eee None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:39.626231 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 451/903] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:45:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:45:39.632187 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ace5d00-a543-4f1e-99b8-47c32f508fa4 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:39.633005 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ace5d00-a543-4f1e-99b8-47c32f508fa4 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:39.633485 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ace5d00-a543-4f1e-99b8-47c32f508fa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:39.650695 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-2ace5d00-a543-4f1e-99b8-47c32f508fa4 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:45:39.652309 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2ace5d00-a543-4f1e-99b8-47c32f508fa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:45:39.723813 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 453/904] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:45:39 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 10 12:45:39.750308 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce80906e-32f4-4d28-b089-d553a62d5eee None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:39.800144 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:39.870497 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce80906e-32f4-4d28-b089-d553a62d5eee None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:39.906375 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2567104a-d15e-42c8-b802-8c18f9f9c82a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:39.907528 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2567104a-d15e-42c8-b802-8c18f9f9c82a None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:39.908049 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2567104a-d15e-42c8-b802-8c18f9f9c82a None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:39.908543 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2567104a-d15e-42c8-b802-8c18f9f9c82a None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:39.909428 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2567104a-d15e-42c8-b802-8c18f9f9c82a None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:39.910989 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2567104a-d15e-42c8-b802-8c18f9f9c82a None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:39.924774 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-2567104a-d15e-42c8-b802-8c18f9f9c82a None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 10 12:45:39.924774 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:39.924774 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 10 12:45:39.924774 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 10 12:45:39.924774 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 10 12:45:39.924774 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 10 12:45:39.924774 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 10 12:45:39.924774 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 10 12:45:39.924774 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 10 12:45:39.924774 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:39.924774 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 10 12:45:39.924774 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:39.924774 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:39.924774 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:39.924774 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:39.924774 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:39.925740 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:39.925740 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:39.925740 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:39.925740 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:39.925740 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:39.925740 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:39.925740 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:39.925740 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 10 12:45:39.925740 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 10 12:45:39.925740 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 10 12:45:39.925740 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 10 12:45:39.925740 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:39.925740 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:39.925740 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:39.925740 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:39.925740 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:39.926593 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:39.926593 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:39.926593 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:39.926593 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:39.926593 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:39.926593 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:39.926593 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:39.926593 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:39.926593 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:39.926593 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:39.926593 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:39.926593 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 10 12:45:39.926593 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 10 12:45:39.926593 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 10 12:45:39.926593 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:39.930067 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 452/905] 10.222.0.27 () {56 vars in 1132 bytes} [Fri Jun 10 12:45:39 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:39.937260 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ace5d00-a543-4f1e-99b8-47c32f508fa4 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:40.045264 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ace5d00-a543-4f1e-99b8-47c32f508fa4 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:40.057461 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:40.083776 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f064d8d-036c-4f64-ab40-38be88be1084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:40.085485 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f064d8d-036c-4f64-ab40-38be88be1084 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:40.086039 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f064d8d-036c-4f64-ab40-38be88be1084 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:40.086523 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f064d8d-036c-4f64-ab40-38be88be1084 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:40.087571 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f064d8d-036c-4f64-ab40-38be88be1084 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:40.088173 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f064d8d-036c-4f64-ab40-38be88be1084 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:40.089828 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f064d8d-036c-4f64-ab40-38be88be1084 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:40.096088 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 454/906] 10.222.0.27 () {56 vars in 1126 bytes} [Fri Jun 10 12:45:39 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:40.102083 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2567104a-d15e-42c8-b802-8c18f9f9c82a None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:40.147854 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ccc264-9bba-481e-9431-1b54c0907b78 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:40.148520 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ccc264-9bba-481e-9431-1b54c0907b78 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:40.148807 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ccc264-9bba-481e-9431-1b54c0907b78 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:40.150619 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 455/907] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:45:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:45:40.156871 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41976b6a-bada-4253-855c-99ba38505004 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:40.157101 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41976b6a-bada-4253-855c-99ba38505004 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:40.157383 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41976b6a-bada-4253-855c-99ba38505004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:40.180314 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-41976b6a-bada-4253-855c-99ba38505004 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:45:40.181840 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-41976b6a-bada-4253-855c-99ba38505004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:45:40.217290 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2567104a-d15e-42c8-b802-8c18f9f9c82a None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:40.229047 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:40.259885 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 456/908] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:45:40 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 10 12:45:40.266400 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59998ad4-d393-4d24-a1c4-d342b6ef8ddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:40.267065 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59998ad4-d393-4d24-a1c4-d342b6ef8ddb None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:40.267353 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59998ad4-d393-4d24-a1c4-d342b6ef8ddb None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:40.267630 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59998ad4-d393-4d24-a1c4-d342b6ef8ddb None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:40.268372 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59998ad4-d393-4d24-a1c4-d342b6ef8ddb None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:40.268681 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59998ad4-d393-4d24-a1c4-d342b6ef8ddb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:40.270049 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59998ad4-d393-4d24-a1c4-d342b6ef8ddb None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:40.276053 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 453/909] 10.222.0.27 () {56 vars in 1126 bytes} [Fri Jun 10 12:45:40 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:40.281551 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59998ad4-d393-4d24-a1c4-d342b6ef8ddb None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:40.281707 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41976b6a-bada-4253-855c-99ba38505004 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:40.390322 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59998ad4-d393-4d24-a1c4-d342b6ef8ddb None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:40.403564 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41976b6a-bada-4253-855c-99ba38505004 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:40.406701 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:40.434493 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c404433c-06d6-45d6-87d9-fe435d974525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:40.435617 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c404433c-06d6-45d6-87d9-fe435d974525 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:40.436075 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c404433c-06d6-45d6-87d9-fe435d974525 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:40.436636 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c404433c-06d6-45d6-87d9-fe435d974525 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:40.438251 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c404433c-06d6-45d6-87d9-fe435d974525 None admin] RBAC: Adding query filter params (interface=public, service_id=c52bd74916bc455dac5438d4540222ad, region_id=RegionOne) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:40.438812 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c404433c-06d6-45d6-87d9-fe435d974525 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:40.441088 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c404433c-06d6-45d6-87d9-fe435d974525 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:40.443007 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d053e7ed-876c-43c2-8acb-fed45080758d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:40.443807 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d053e7ed-876c-43c2-8acb-fed45080758d None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:40.444507 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d053e7ed-876c-43c2-8acb-fed45080758d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:40.444669 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d053e7ed-876c-43c2-8acb-fed45080758d None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:40.445534 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d053e7ed-876c-43c2-8acb-fed45080758d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:40.447032 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d053e7ed-876c-43c2-8acb-fed45080758d None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:40.448060 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 454/910] 10.222.0.27 () {56 vars in 1251 bytes} [Fri Jun 10 12:45:40 2022] GET /identity/v3/endpoints?service_id=c52bd74916bc455dac5438d4540222ad&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:40.453307 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c404433c-06d6-45d6-87d9-fe435d974525 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:40.490160 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:40.532596 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 457/911] 10.222.0.27 () {60 vars in 1141 bytes} [Fri Jun 10 12:45:40 2022] POST /identity/v3/services => generated 241 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 12:45:40.567397 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c404433c-06d6-45d6-87d9-fe435d974525 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:40.615381 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9e93eb-16e9-4272-aaf5-0dad7575ee12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:40.616099 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9e93eb-16e9-4272-aaf5-0dad7575ee12 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:40.616382 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9e93eb-16e9-4272-aaf5-0dad7575ee12 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:40.616661 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9e93eb-16e9-4272-aaf5-0dad7575ee12 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:40.617362 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9e93eb-16e9-4272-aaf5-0dad7575ee12 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:40.618781 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9e93eb-16e9-4272-aaf5-0dad7575ee12 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:40.626149 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 455/912] 10.222.0.27 () {56 vars in 1087 bytes} [Fri Jun 10 12:45:40 2022] GET /identity/v3/services => generated 1497 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 10 12:45:40.930224 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c03f9f0-6879-47b9-875d-13fae9530ed7 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:40.931186 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c03f9f0-6879-47b9-875d-13fae9530ed7 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:40.931690 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c03f9f0-6879-47b9-875d-13fae9530ed7 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:40.933591 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 458/913] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:45:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:45:40.939145 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acb2901-bdec-4ac1-b6fe-86d4b20f913c None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:40.939704 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acb2901-bdec-4ac1-b6fe-86d4b20f913c None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:40.940278 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acb2901-bdec-4ac1-b6fe-86d4b20f913c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:40.958300 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-1acb2901-bdec-4ac1-b6fe-86d4b20f913c None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:45:40.959883 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-1acb2901-bdec-4ac1-b6fe-86d4b20f913c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:45:41.044312 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 456/914] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:45:40 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 10 12:45:41.063414 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c03f9f0-6879-47b9-875d-13fae9530ed7 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:41.152218 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c470bc-c4be-47f1-8f85-9ef96fba31d5 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:41.152218 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c470bc-c4be-47f1-8f85-9ef96fba31d5 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:41.152218 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c470bc-c4be-47f1-8f85-9ef96fba31d5 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:41.152716 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 457/915] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:45:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 10 12:45:41.157343 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaceb48-9cf8-4fe9-b3fb-6f8c1d13b14a None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:41.157626 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaceb48-9cf8-4fe9-b3fb-6f8c1d13b14a None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:41.157911 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaceb48-9cf8-4fe9-b3fb-6f8c1d13b14a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:41.177116 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-efaceb48-9cf8-4fe9-b3fb-6f8c1d13b14a None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:45:41.179228 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-efaceb48-9cf8-4fe9-b3fb-6f8c1d13b14a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:45:41.181075 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c03f9f0-6879-47b9-875d-13fae9530ed7 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:41.207267 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:41.224572 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfe9a04-3851-4db5-abab-1a25033387bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:41.226249 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfe9a04-3851-4db5-abab-1a25033387bb None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:41.226750 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfe9a04-3851-4db5-abab-1a25033387bb None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:41.227220 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfe9a04-3851-4db5-abab-1a25033387bb None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:41.228296 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfe9a04-3851-4db5-abab-1a25033387bb None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:41.230317 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfe9a04-3851-4db5-abab-1a25033387bb None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:41.248166 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-3dfe9a04-3851-4db5-abab-1a25033387bb None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 10 12:45:41.248166 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:41.248166 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 10 12:45:41.248166 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 10 12:45:41.248166 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 10 12:45:41.248166 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 10 12:45:41.248166 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 10 12:45:41.248166 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 10 12:45:41.248166 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 10 12:45:41.248166 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:41.248166 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 10 12:45:41.248166 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:41.248166 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:41.248166 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:41.248166 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:41.248166 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:41.248166 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:41.249186 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:41.249186 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:41.249186 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:41.249186 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:41.249186 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:41.249186 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:41.249186 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 10 12:45:41.249186 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 10 12:45:41.249186 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 10 12:45:41.249186 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 10 12:45:41.249186 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:41.249186 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:41.249186 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:41.249186 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:41.249186 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:41.249186 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:41.250040 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:41.250040 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:41.250040 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:41.250040 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:41.250040 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:41.250040 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:41.250040 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:41.250040 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:41.250040 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:41.250040 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:41.250040 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 10 12:45:41.250040 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 10 12:45:41.250040 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 10 12:45:41.250040 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:41.255975 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 459/916] 10.222.0.27 () {56 vars in 1114 bytes} [Fri Jun 10 12:45:41 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:41.260201 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfe9a04-3851-4db5-abab-1a25033387bb None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:41.275881 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 458/917] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:45:41 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 10 12:45:41.290458 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaceb48-9cf8-4fe9-b3fb-6f8c1d13b14a None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:41.292025 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:41.370281 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfe9a04-3851-4db5-abab-1a25033387bb None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:41.406576 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bc99ae-5a21-48f8-a2df-9ae59a0a389a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:41.407559 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bc99ae-5a21-48f8-a2df-9ae59a0a389a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:41.408100 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bc99ae-5a21-48f8-a2df-9ae59a0a389a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:41.408594 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bc99ae-5a21-48f8-a2df-9ae59a0a389a None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:41.409499 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bc99ae-5a21-48f8-a2df-9ae59a0a389a None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:41.410006 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bc99ae-5a21-48f8-a2df-9ae59a0a389a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:41.411832 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bc99ae-5a21-48f8-a2df-9ae59a0a389a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:41.417965 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 460/918] 10.222.0.27 () {56 vars in 1114 bytes} [Fri Jun 10 12:45:41 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:41.424954 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bc99ae-5a21-48f8-a2df-9ae59a0a389a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:41.425761 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaceb48-9cf8-4fe9-b3fb-6f8c1d13b14a None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:41.471169 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c2e60e-f656-4d50-9fac-95a689f8d9a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:41.472069 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c2e60e-f656-4d50-9fac-95a689f8d9a1 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:41.472351 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c2e60e-f656-4d50-9fac-95a689f8d9a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:41.472621 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c2e60e-f656-4d50-9fac-95a689f8d9a1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:41.486652 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c2e60e-f656-4d50-9fac-95a689f8d9a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:45:41.498147 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-25c2e60e-f656-4d50-9fac-95a689f8d9a1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 10 12:45:41.498147 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:41.498147 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:41.498147 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:41.498147 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:41.498147 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:41.498147 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:41.498147 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:41.498147 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:41.498147 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:41.498147 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:41.498147 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:41.498147 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:45:41.498147 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:45:41.498147 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:45:41.498147 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:45:41.499110 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:45:41.499110 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:45:41.499110 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:45:41.499110 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:45:41.499110 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:41.499110 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:41.499110 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:41.499110 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:41.499110 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:41.499110 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:41.499110 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:41.499110 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:41.499110 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:41.499110 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:41.499110 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:41.499110 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:41.499990 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:41.499990 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:41.499990 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:41.499990 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:41.499990 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:45:41.499990 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:45:41.499990 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:45:41.499990 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:45:41.499990 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:45:41.499990 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:45:41.499990 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 10 12:45:41.499990 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:41.504760 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 459/919] 10.222.0.27 () {56 vars in 1120 bytes} [Fri Jun 10 12:45:41 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 10 12:45:41.506369 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c2e60e-f656-4d50-9fac-95a689f8d9a1 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:41.530372 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:41.546245 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:41.557393 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bc99ae-5a21-48f8-a2df-9ae59a0a389a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:41.603453 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae8e16a-312f-42d2-80cc-e736c105cb04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:41.604155 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae8e16a-312f-42d2-80cc-e736c105cb04 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:41.604480 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae8e16a-312f-42d2-80cc-e736c105cb04 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:41.604763 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae8e16a-312f-42d2-80cc-e736c105cb04 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:41.605468 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae8e16a-312f-42d2-80cc-e736c105cb04 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:41.605773 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae8e16a-312f-42d2-80cc-e736c105cb04 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:41.607126 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae8e16a-312f-42d2-80cc-e736c105cb04 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:41.616855 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 461/920] 10.222.0.27 () {56 vars in 1114 bytes} [Fri Jun 10 12:45:41 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 12:45:41.618057 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae8e16a-312f-42d2-80cc-e736c105cb04 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:41.632866 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c2e60e-f656-4d50-9fac-95a689f8d9a1 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:41.671398 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2ce63f-a907-4155-a9bc-e062b8dc3092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:41.672456 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2ce63f-a907-4155-a9bc-e062b8dc3092 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:41.672946 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2ce63f-a907-4155-a9bc-e062b8dc3092 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:41.673435 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2ce63f-a907-4155-a9bc-e062b8dc3092 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:41.674378 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2ce63f-a907-4155-a9bc-e062b8dc3092 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:41.674884 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2ce63f-a907-4155-a9bc-e062b8dc3092 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:41.676506 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2ce63f-a907-4155-a9bc-e062b8dc3092 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:41.685750 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 460/921] 10.222.0.27 () {56 vars in 1115 bytes} [Fri Jun 10 12:45:41 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 12:45:41.691210 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2ce63f-a907-4155-a9bc-e062b8dc3092 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:41.730536 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae8e16a-312f-42d2-80cc-e736c105cb04 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:41.770151 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528b1034-42f0-4692-8b64-07897d4a5c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:41.771139 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528b1034-42f0-4692-8b64-07897d4a5c08 None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:41.772151 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528b1034-42f0-4692-8b64-07897d4a5c08 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:41.772571 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528b1034-42f0-4692-8b64-07897d4a5c08 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:41.773458 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528b1034-42f0-4692-8b64-07897d4a5c08 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:41.774954 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528b1034-42f0-4692-8b64-07897d4a5c08 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:41.798813 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:41.819658 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2ce63f-a907-4155-a9bc-e062b8dc3092 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:41.864658 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 462/922] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Jun 10 12:45:41 2022] POST /identity/v3/endpoints => generated 346 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 12:45:41.875916 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46025036-ecd5-4e01-916b-d236cdf1e672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:41.875916 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46025036-ecd5-4e01-916b-d236cdf1e672 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:41.875916 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46025036-ecd5-4e01-916b-d236cdf1e672 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:41.875916 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46025036-ecd5-4e01-916b-d236cdf1e672 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:41.887329 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46025036-ecd5-4e01-916b-d236cdf1e672 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:41.889251 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46025036-ecd5-4e01-916b-d236cdf1e672 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:41.910284 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-46025036-ecd5-4e01-916b-d236cdf1e672 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:41.910284 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:41.910284 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:41.910284 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:41.910284 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:41.910284 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:41.910284 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:41.910284 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:41.910284 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:41.910284 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:41.910284 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:41.910284 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:41.910284 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 10 12:45:41.910284 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 10 12:45:41.910284 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 10 12:45:41.911748 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 10 12:45:41.911748 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:41.911748 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:41.911748 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:41.911748 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:41.911748 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:41.911748 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:41.911748 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:41.911748 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:41.911748 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:41.911748 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:41.911748 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:41.911748 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:41.911748 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:41.911748 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:41.911748 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:41.911748 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:41.913687 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 10 12:45:41.913687 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 10 12:45:41.913687 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:41.913687 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:41.920091 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 461/923] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:45:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 10 12:45:41.922242 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528b1034-42f0-4692-8b64-07897d4a5c08 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:41.941338 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:42.037940 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528b1034-42f0-4692-8b64-07897d4a5c08 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:42.075642 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2cc22d-a5f8-4bfd-b083-27726617168d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:42.076338 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2cc22d-a5f8-4bfd-b083-27726617168d None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:42.076621 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2cc22d-a5f8-4bfd-b083-27726617168d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:42.077026 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2cc22d-a5f8-4bfd-b083-27726617168d None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:42.077827 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2cc22d-a5f8-4bfd-b083-27726617168d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:42.078138 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2cc22d-a5f8-4bfd-b083-27726617168d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:42.079506 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2cc22d-a5f8-4bfd-b083-27726617168d None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:42.092658 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 463/924] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Jun 10 12:45:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:42.098494 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46025036-ecd5-4e01-916b-d236cdf1e672 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:42.212625 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46025036-ecd5-4e01-916b-d236cdf1e672 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:42.239401 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:42.249278 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4f5909-afda-4409-870d-f22f89c028c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:42.250095 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f5909-afda-4409-870d-f22f89c028c7 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:42.250375 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f5909-afda-4409-870d-f22f89c028c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:42.250748 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f5909-afda-4409-870d-f22f89c028c7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:42.259553 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4f5909-afda-4409-870d-f22f89c028c7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:42.261289 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4f5909-afda-4409-870d-f22f89c028c7 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:42.276966 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-0f4f5909-afda-4409-870d-f22f89c028c7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 10 12:45:42.276966 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:42.276966 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:42.276966 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:42.276966 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:42.276966 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:42.276966 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:42.276966 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:42.276966 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:42.276966 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:42.276966 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:42.276966 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:42.276966 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 10 12:45:42.276966 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 10 12:45:42.276966 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 10 12:45:42.276966 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 10 12:45:42.278238 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:42.278238 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:42.278238 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 10 12:45:42.278238 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 10 12:45:42.278238 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 10 12:45:42.278238 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 10 12:45:42.278238 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:42.278238 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:42.278238 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:42.278238 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:42.278238 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:42.278238 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:42.278238 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:42.278238 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:42.278238 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:42.278238 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:42.278238 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:42.278238 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:42.279214 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:42.279214 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:42.279214 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 10 12:45:42.279214 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 10 12:45:42.279214 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 10 12:45:42.279214 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 10 12:45:42.279214 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 10 12:45:42.279214 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 10 12:45:42.279214 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 10 12:45:42.279214 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:42.279214 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 462/925] 10.222.0.27 () {56 vars in 1117 bytes} [Fri Jun 10 12:45:42 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:42.286345 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2cc22d-a5f8-4bfd-b083-27726617168d None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:42.373229 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:42.405630 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2cc22d-a5f8-4bfd-b083-27726617168d None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:42.443088 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ba289f-a025-48b6-a022-fe367c6fe7c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:42.444747 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ba289f-a025-48b6-a022-fe367c6fe7c9 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:42.445260 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ba289f-a025-48b6-a022-fe367c6fe7c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:42.445675 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ba289f-a025-48b6-a022-fe367c6fe7c9 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:42.446650 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ba289f-a025-48b6-a022-fe367c6fe7c9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:42.447098 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ba289f-a025-48b6-a022-fe367c6fe7c9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:42.448696 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ba289f-a025-48b6-a022-fe367c6fe7c9 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:42.472278 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 464/926] 10.222.0.27 () {56 vars in 1149 bytes} [Fri Jun 10 12:45:42 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:42.473870 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4f5909-afda-4409-870d-f22f89c028c7 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:42.607506 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4f5909-afda-4409-870d-f22f89c028c7 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:42.653391 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09923382-5ae9-4f58-889d-aa983b5661b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:42.654295 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09923382-5ae9-4f58-889d-aa983b5661b0 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:42.654639 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09923382-5ae9-4f58-889d-aa983b5661b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:42.654921 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09923382-5ae9-4f58-889d-aa983b5661b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:42.672827 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09923382-5ae9-4f58-889d-aa983b5661b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:42.674342 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09923382-5ae9-4f58-889d-aa983b5661b0 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:42.697897 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-09923382-5ae9-4f58-889d-aa983b5661b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:42.697897 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:42.697897 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:42.697897 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:42.697897 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:42.697897 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:42.697897 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:42.697897 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:42.697897 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:42.697897 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:42.697897 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:42.697897 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:42.697897 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 10 12:45:42.697897 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 10 12:45:42.697897 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 10 12:45:42.698861 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 10 12:45:42.698861 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:42.698861 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:42.698861 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:42.698861 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:42.698861 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:42.698861 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:42.698861 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:42.698861 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:42.698861 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:42.698861 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:42.698861 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:42.698861 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:42.698861 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:42.698861 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:42.698861 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:42.698861 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:42.699768 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 10 12:45:42.699768 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 10 12:45:42.699768 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:42.699768 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:42.699768 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 463/927] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:45:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:42.706328 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ba289f-a025-48b6-a022-fe367c6fe7c9 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:42.792197 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cc9111-5fc8-46aa-ac28-9adc95685311 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:42.792408 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cc9111-5fc8-46aa-ac28-9adc95685311 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:42.792675 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cc9111-5fc8-46aa-ac28-9adc95685311 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:42.794331 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 464/928] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:45:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:45:42.800900 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921ec32f-fd59-45d1-aa99-8431d9647f4b None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:42.801237 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921ec32f-fd59-45d1-aa99-8431d9647f4b None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:42.801485 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921ec32f-fd59-45d1-aa99-8431d9647f4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:42.830838 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-921ec32f-fd59-45d1-aa99-8431d9647f4b None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:45:42.832483 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-921ec32f-fd59-45d1-aa99-8431d9647f4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:45:42.835058 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ba289f-a025-48b6-a022-fe367c6fe7c9 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:42.865628 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:42.873851 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:42.880210 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34c52a0-3582-4fdf-80b0-e8ca7846524e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:42.881432 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34c52a0-3582-4fdf-80b0-e8ca7846524e None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:42.881720 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34c52a0-3582-4fdf-80b0-e8ca7846524e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:42.882003 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34c52a0-3582-4fdf-80b0-e8ca7846524e None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:42.886673 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34c52a0-3582-4fdf-80b0-e8ca7846524e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:42.886673 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34c52a0-3582-4fdf-80b0-e8ca7846524e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:42.886673 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34c52a0-3582-4fdf-80b0-e8ca7846524e None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:42.896462 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 465/929] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Jun 10 12:45:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:42.898259 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f76dc3-4f44-4fba-8cc2-b6ee48047fe9 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:42.898552 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f76dc3-4f44-4fba-8cc2-b6ee48047fe9 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:42.898841 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f76dc3-4f44-4fba-8cc2-b6ee48047fe9 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:42.903909 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 466/930] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:45:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:45:42.903909 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f76dc3-4f44-4fba-8cc2-b6ee48047fe9 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:42.914402 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 465/931] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:45:42 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 10 12:45:42.915961 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b82a711-0034-471e-9104-f4c3aa747bc2 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:42.916437 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b82a711-0034-471e-9104-f4c3aa747bc2 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:42.916893 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b82a711-0034-471e-9104-f4c3aa747bc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:42.939668 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-4b82a711-0034-471e-9104-f4c3aa747bc2 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:45:42.940966 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-4b82a711-0034-471e-9104-f4c3aa747bc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:45:43.006573 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 466/932] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:45:42 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 10 12:45:43.008152 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b82a711-0034-471e-9104-f4c3aa747bc2 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:43.010872 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f76dc3-4f44-4fba-8cc2-b6ee48047fe9 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:43.046225 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692fab14-e7f2-4564-b382-162e67cb1d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:43.046942 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692fab14-e7f2-4564-b382-162e67cb1d19 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:43.047229 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692fab14-e7f2-4564-b382-162e67cb1d19 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:43.047507 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692fab14-e7f2-4564-b382-162e67cb1d19 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:43.054310 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692fab14-e7f2-4564-b382-162e67cb1d19 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:43.055742 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692fab14-e7f2-4564-b382-162e67cb1d19 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:43.070322 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-692fab14-e7f2-4564-b382-162e67cb1d19 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 10 12:45:43.070322 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:43.070322 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:43.070322 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:43.070322 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:43.070322 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:43.070322 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:43.070322 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:43.070322 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:43.070322 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:43.070322 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:43.070322 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:43.070322 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 10 12:45:43.070322 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 10 12:45:43.070322 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 10 12:45:43.071231 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 10 12:45:43.071231 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:43.071231 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:43.071231 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:43.071231 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:43.071231 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:43.071231 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:43.071231 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:43.071231 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:43.071231 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:43.071231 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:43.071231 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:43.071231 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:43.071231 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:43.071231 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:43.071231 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:43.071231 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:43.072843 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 10 12:45:43.072843 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 10 12:45:43.072843 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 10 12:45:43.072843 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 10 12:45:43.072843 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 10 12:45:43.072843 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 10 12:45:43.072843 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 10 12:45:43.072843 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:43.072843 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 467/933] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Jun 10 12:45:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:43.072843 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692fab14-e7f2-4564-b382-162e67cb1d19 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:43.118129 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:43.122053 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:43.122965 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b82a711-0034-471e-9104-f4c3aa747bc2 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:43.164209 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d42e05-f6d9-45ff-8e0c-169f08a53b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:43.165005 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d42e05-f6d9-45ff-8e0c-169f08a53b1f None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:43.165297 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d42e05-f6d9-45ff-8e0c-169f08a53b1f None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:43.165560 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d42e05-f6d9-45ff-8e0c-169f08a53b1f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:43.166228 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d42e05-f6d9-45ff-8e0c-169f08a53b1f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:43.167600 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d42e05-f6d9-45ff-8e0c-169f08a53b1f None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:43.180335 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-d9d42e05-f6d9-45ff-8e0c-169f08a53b1f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 10 12:45:43.180335 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:43.180335 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 10 12:45:43.180335 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 10 12:45:43.180335 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 10 12:45:43.180335 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 10 12:45:43.180335 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 10 12:45:43.180335 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 10 12:45:43.180335 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 10 12:45:43.180335 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:43.180335 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 10 12:45:43.180335 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:43.180335 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:43.180335 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:43.180335 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:43.180335 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:43.180335 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:43.188048 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:43.188048 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:43.188048 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:43.188048 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:43.188048 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:43.188048 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:43.188048 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 10 12:45:43.188048 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 10 12:45:43.188048 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 10 12:45:43.188048 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 10 12:45:43.188048 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:43.188048 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:43.188048 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:43.188048 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:43.188048 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:43.188048 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:43.188922 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:43.188922 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:43.188922 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:43.188922 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:43.188922 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:43.188922 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:43.188922 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:43.188922 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:43.188922 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:43.188922 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:43.188922 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 10 12:45:43.188922 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 10 12:45:43.188922 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 10 12:45:43.188922 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:43.189685 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692fab14-e7f2-4564-b382-162e67cb1d19 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:43.189761 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 467/934] 10.222.0.27 () {56 vars in 1126 bytes} [Fri Jun 10 12:45:43 2022] GET /identity/v3/services/object-store => generated 93 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:43.189761 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d42e05-f6d9-45ff-8e0c-169f08a53b1f None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:43.220365 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f0234a-2dd1-4c9a-9bb4-7405be4b62ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:43.221187 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f0234a-2dd1-4c9a-9bb4-7405be4b62ab None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:43.221474 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f0234a-2dd1-4c9a-9bb4-7405be4b62ab None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:43.221757 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f0234a-2dd1-4c9a-9bb4-7405be4b62ab None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:43.222421 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f0234a-2dd1-4c9a-9bb4-7405be4b62ab None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:43.223793 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f0234a-2dd1-4c9a-9bb4-7405be4b62ab None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:43.236642 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-70f0234a-2dd1-4c9a-9bb4-7405be4b62ab None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 10 12:45:43.236642 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:43.236642 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 10 12:45:43.236642 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 10 12:45:43.236642 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 10 12:45:43.236642 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 10 12:45:43.236642 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 10 12:45:43.236642 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 10 12:45:43.236642 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 10 12:45:43.236642 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:43.236642 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 10 12:45:43.236642 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:43.236642 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:43.236642 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:43.236642 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:43.236642 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:43.237639 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:43.237639 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:43.237639 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:43.237639 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:43.237639 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:43.237639 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:43.237639 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:43.237639 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 10 12:45:43.237639 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 10 12:45:43.237639 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 10 12:45:43.237639 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 10 12:45:43.237639 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:43.237639 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:43.237639 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:43.237639 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:43.237639 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:43.238502 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:43.238502 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:43.238502 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:43.238502 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:43.238502 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:43.238502 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:43.238502 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:43.238502 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:43.238502 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:43.238502 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:43.238502 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:43.238502 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 10 12:45:43.238502 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 10 12:45:43.238502 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 10 12:45:43.238502 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:43.238502 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 468/935] 10.222.0.27 () {56 vars in 1132 bytes} [Fri Jun 10 12:45:43 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:43.239365 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f0234a-2dd1-4c9a-9bb4-7405be4b62ab None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:43.308808 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d42e05-f6d9-45ff-8e0c-169f08a53b1f None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:43.344195 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde5f09b-01d0-4c2e-8ff8-ce4f901c4a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:43.344851 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde5f09b-01d0-4c2e-8ff8-ce4f901c4a8b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:43.345130 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde5f09b-01d0-4c2e-8ff8-ce4f901c4a8b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:43.345296 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f0234a-2dd1-4c9a-9bb4-7405be4b62ab None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:43.345421 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde5f09b-01d0-4c2e-8ff8-ce4f901c4a8b None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:43.346245 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde5f09b-01d0-4c2e-8ff8-ce4f901c4a8b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:43.346552 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde5f09b-01d0-4c2e-8ff8-ce4f901c4a8b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:43.348009 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde5f09b-01d0-4c2e-8ff8-ce4f901c4a8b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:43.360164 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:43.366508 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 468/936] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Jun 10 12:45:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:43.366508 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde5f09b-01d0-4c2e-8ff8-ce4f901c4a8b None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:43.385823 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52e6a8d-3ded-45fe-80c6-c7ba53bc7548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:43.386485 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52e6a8d-3ded-45fe-80c6-c7ba53bc7548 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:43.386764 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52e6a8d-3ded-45fe-80c6-c7ba53bc7548 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:43.387044 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52e6a8d-3ded-45fe-80c6-c7ba53bc7548 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:43.387745 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52e6a8d-3ded-45fe-80c6-c7ba53bc7548 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:43.388073 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52e6a8d-3ded-45fe-80c6-c7ba53bc7548 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:43.389421 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52e6a8d-3ded-45fe-80c6-c7ba53bc7548 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:43.389975 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:43.395762 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 469/937] 10.222.0.27 () {56 vars in 1122 bytes} [Fri Jun 10 12:45:43 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:43.396603 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52e6a8d-3ded-45fe-80c6-c7ba53bc7548 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:43.475316 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde5f09b-01d0-4c2e-8ff8-ce4f901c4a8b None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:43.501003 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52e6a8d-3ded-45fe-80c6-c7ba53bc7548 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:43.510881 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16c4e34-2f2c-4f5a-a56b-8f9264296b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:43.511550 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16c4e34-2f2c-4f5a-a56b-8f9264296b94 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:43.512172 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16c4e34-2f2c-4f5a-a56b-8f9264296b94 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:43.512460 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16c4e34-2f2c-4f5a-a56b-8f9264296b94 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:43.513490 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16c4e34-2f2c-4f5a-a56b-8f9264296b94 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:43.513800 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16c4e34-2f2c-4f5a-a56b-8f9264296b94 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:43.515709 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16c4e34-2f2c-4f5a-a56b-8f9264296b94 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:43.520704 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 469/938] 10.222.0.27 () {56 vars in 1126 bytes} [Fri Jun 10 12:45:43 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:43.525816 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16c4e34-2f2c-4f5a-a56b-8f9264296b94 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:43.538098 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1378f87c-4f68-47d6-9560-2201ace78e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:43.538991 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1378f87c-4f68-47d6-9560-2201ace78e91 None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:43.539273 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1378f87c-4f68-47d6-9560-2201ace78e91 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:43.539548 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1378f87c-4f68-47d6-9560-2201ace78e91 None admin] PATH_INFO: `/v3/projects/73ea5b23a2d34544ab493f4c5e4034b8/users/da66362070694e91bbe2cfbbd161b8a7/roles/55ae4712a75e4f9bb542575264d55f64` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:43.542448 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1378f87c-4f68-47d6-9560-2201ace78e91 None admin] RBAC: Authorizing `identity:create_grant(project_id=73ea5b23a2d34544ab493f4c5e4034b8, user_id=da66362070694e91bbe2cfbbd161b8a7, role_id=55ae4712a75e4f9bb542575264d55f64)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:43.543853 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1378f87c-4f68-47d6-9560-2201ace78e91 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:43.612205 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:43.612619 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 470/939] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Jun 10 12:45:43 2022] PUT /identity/v3/projects/73ea5b23a2d34544ab493f4c5e4034b8/users/da66362070694e91bbe2cfbbd161b8a7/roles/55ae4712a75e4f9bb542575264d55f64 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 12:45:43.613967 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1378f87c-4f68-47d6-9560-2201ace78e91 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:43.638124 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16c4e34-2f2c-4f5a-a56b-8f9264296b94 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:43.666336 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:43.691390 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153f3705-7ef8-479e-81d3-bc61443dbf3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:43.693707 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f3705-7ef8-479e-81d3-bc61443dbf3f None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:43.694488 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f3705-7ef8-479e-81d3-bc61443dbf3f None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:43.694921 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f3705-7ef8-479e-81d3-bc61443dbf3f None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:43.695810 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153f3705-7ef8-479e-81d3-bc61443dbf3f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:43.696443 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153f3705-7ef8-479e-81d3-bc61443dbf3f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:43.698732 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153f3705-7ef8-479e-81d3-bc61443dbf3f None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:43.706413 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 470/940] 10.222.0.27 () {56 vars in 1122 bytes} [Fri Jun 10 12:45:43 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:43.712115 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153f3705-7ef8-479e-81d3-bc61443dbf3f None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:43.744812 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1378f87c-4f68-47d6-9560-2201ace78e91 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:43.785664 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96df56c-b193-43cf-93b7-792204ad3ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:43.786915 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96df56c-b193-43cf-93b7-792204ad3ae5 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:43.787781 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96df56c-b193-43cf-93b7-792204ad3ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:43.788109 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96df56c-b193-43cf-93b7-792204ad3ae5 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:43.789145 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96df56c-b193-43cf-93b7-792204ad3ae5 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:43.789458 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96df56c-b193-43cf-93b7-792204ad3ae5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:43.790834 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96df56c-b193-43cf-93b7-792204ad3ae5 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:43.798368 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 471/941] 10.222.0.27 () {56 vars in 1126 bytes} [Fri Jun 10 12:45:43 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:43.804491 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96df56c-b193-43cf-93b7-792204ad3ae5 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:43.846953 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153f3705-7ef8-479e-81d3-bc61443dbf3f None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:43.889651 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834bc170-26f3-410c-b8f1-29f73341f91c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:43.890377 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834bc170-26f3-410c-b8f1-29f73341f91c None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:43.890658 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834bc170-26f3-410c-b8f1-29f73341f91c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:43.890933 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834bc170-26f3-410c-b8f1-29f73341f91c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:43.891799 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834bc170-26f3-410c-b8f1-29f73341f91c None admin] RBAC: Adding query filter params (interface=public, service_id=8deed0a7aca34f638267dc35e549e9ad, region_id=RegionOne) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:43.892149 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834bc170-26f3-410c-b8f1-29f73341f91c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:43.893525 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834bc170-26f3-410c-b8f1-29f73341f91c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:43.899607 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 471/942] 10.222.0.27 () {56 vars in 1251 bytes} [Fri Jun 10 12:45:43 2022] GET /identity/v3/endpoints?service_id=8deed0a7aca34f638267dc35e549e9ad&interface=public®ion_id=RegionOne => generated 194 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:43.904840 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834bc170-26f3-410c-b8f1-29f73341f91c None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:43.927977 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:43.937517 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96df56c-b193-43cf-93b7-792204ad3ae5 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:43.939396 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:43.977437 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679668c6-c8ff-47db-8f88-f37e551a3607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:43.978673 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679668c6-c8ff-47db-8f88-f37e551a3607 None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:43.978673 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679668c6-c8ff-47db-8f88-f37e551a3607 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:43.978981 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679668c6-c8ff-47db-8f88-f37e551a3607 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:43.979837 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679668c6-c8ff-47db-8f88-f37e551a3607 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:43.981343 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679668c6-c8ff-47db-8f88-f37e551a3607 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:44.030457 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834bc170-26f3-410c-b8f1-29f73341f91c None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:44.068107 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 472/943] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Jun 10 12:45:43 2022] POST /identity/v3/endpoints => generated 347 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 12:45:44.080036 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b13869-4f38-4c2e-9780-cfd3b99fb36b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:44.081051 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b13869-4f38-4c2e-9780-cfd3b99fb36b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:44.081557 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b13869-4f38-4c2e-9780-cfd3b99fb36b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:44.082068 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b13869-4f38-4c2e-9780-cfd3b99fb36b None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:44.082965 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b13869-4f38-4c2e-9780-cfd3b99fb36b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:44.085350 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b13869-4f38-4c2e-9780-cfd3b99fb36b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:44.093451 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 472/944] 10.222.0.27 () {56 vars in 1087 bytes} [Fri Jun 10 12:45:43 2022] GET /identity/v3/services => generated 1497 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 12:45:44.243976 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a7fc95-062a-48f9-95b7-35b779ed68b2 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:44.244660 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a7fc95-062a-48f9-95b7-35b779ed68b2 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:44.245151 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a7fc95-062a-48f9-95b7-35b779ed68b2 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:44.247137 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 473/945] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:45:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:45:44.253154 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35457ffa-a117-4dbb-b1a4-dd76ed24e36b None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:44.253777 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35457ffa-a117-4dbb-b1a4-dd76ed24e36b None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:44.254257 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35457ffa-a117-4dbb-b1a4-dd76ed24e36b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:44.273581 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-35457ffa-a117-4dbb-b1a4-dd76ed24e36b None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:45:44.274792 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-35457ffa-a117-4dbb-b1a4-dd76ed24e36b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:45:44.351208 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 473/946] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:45:44 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 10 12:45:44.374259 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a7fc95-062a-48f9-95b7-35b779ed68b2 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:44.482720 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a7fc95-062a-48f9-95b7-35b779ed68b2 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:44.522721 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf9dac7-eddf-4f8b-bd47-65659347ee3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:44.523308 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf9dac7-eddf-4f8b-bd47-65659347ee3a None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:44.523594 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf9dac7-eddf-4f8b-bd47-65659347ee3a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:44.523887 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf9dac7-eddf-4f8b-bd47-65659347ee3a None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:44.524623 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf9dac7-eddf-4f8b-bd47-65659347ee3a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:44.526052 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf9dac7-eddf-4f8b-bd47-65659347ee3a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:44.527415 dv-cinder-master-3-smb devstack@keystone.service[80237]: WARNING keystone.server.flask.common [None req-daf9dac7-eddf-4f8b-bd47-65659347ee3a None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 10 12:45:44.583229 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.sql.core [None req-daf9dac7-eddf-4f8b-bd47-65659347ee3a None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 10 12:45:44.583229 dv-cinder-master-3-smb devstack@keystone.service[80237]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 10 12:45:44.583229 dv-cinder-master-3-smb devstack@keystone.service[80237]: [parameters: {'id': '8ddf939a3ce84055a13420921dee0166', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 10 12:45:44.583229 dv-cinder-master-3-smb devstack@keystone.service[80237]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80237) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 10 12:45:44.591629 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-daf9dac7-eddf-4f8b-bd47-65659347ee3a None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 8ddf939a3ce84055a13420921dee0166.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 8ddf939a3ce84055a13420921dee0166. Jun 10 12:45:44.591629 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:44.591629 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 10 12:45:44.591629 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 10 12:45:44.591629 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 10 12:45:44.591629 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 10 12:45:44.591629 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 10 12:45:44.591629 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application result = self._query(query) Jun 10 12:45:44.591629 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 10 12:45:44.591629 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application conn.query(q) Jun 10 12:45:44.591629 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 10 12:45:44.591629 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 10 12:45:44.591629 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 10 12:45:44.592474 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application result.read() Jun 10 12:45:44.592474 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 10 12:45:44.592474 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 10 12:45:44.592474 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 10 12:45:44.592474 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 10 12:45:44.592474 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 10 12:45:44.592474 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 10 12:45:44.592474 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 10 12:45:44.592474 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 10 12:45:44.592474 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 10 12:45:44.592474 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:44.592474 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 10 12:45:44.592474 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:44.592474 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:44.592474 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 10 12:45:44.592474 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 10 12:45:44.592474 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 10 12:45:44.592474 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 10 12:45:44.593840 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 10 12:45:44.593840 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application next(self.gen) Jun 10 12:45:44.593840 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 10 12:45:44.593840 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application yield resource Jun 10 12:45:44.593840 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 10 12:45:44.593840 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application next(self.gen) Jun 10 12:45:44.593840 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 10 12:45:44.593840 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application self.session.rollback() Jun 10 12:45:44.593840 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 10 12:45:44.593840 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application self.force_reraise() Jun 10 12:45:44.593840 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 10 12:45:44.593840 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise self.value Jun 10 12:45:44.593840 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 10 12:45:44.593840 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 10 12:45:44.593840 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 10 12:45:44.593840 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application session.commit() Jun 10 12:45:44.593840 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 10 12:45:44.594766 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 10 12:45:44.594766 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 10 12:45:44.594766 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._prepare_impl() Jun 10 12:45:44.594766 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 10 12:45:44.594766 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application self.session.flush() Jun 10 12:45:44.594766 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 10 12:45:44.594766 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._flush(objects) Jun 10 12:45:44.594766 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 10 12:45:44.594766 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 10 12:45:44.594766 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 10 12:45:44.594766 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application compat.raise_( Jun 10 12:45:44.594766 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 10 12:45:44.594766 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception Jun 10 12:45:44.594766 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 10 12:45:44.594766 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application flush_context.execute() Jun 10 12:45:44.594766 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 10 12:45:44.594766 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rec.execute(self) Jun 10 12:45:44.595678 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 10 12:45:44.595678 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 10 12:45:44.595678 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 10 12:45:44.595678 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 10 12:45:44.595678 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 10 12:45:44.595678 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 10 12:45:44.595678 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 10 12:45:44.595678 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 10 12:45:44.595678 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 10 12:45:44.595678 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 10 12:45:44.595678 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 10 12:45:44.595678 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 10 12:45:44.595678 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 10 12:45:44.595678 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 10 12:45:44.595678 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 10 12:45:44.596516 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 10 12:45:44.596516 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 10 12:45:44.596516 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception Jun 10 12:45:44.596516 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 10 12:45:44.596516 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 10 12:45:44.596516 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 10 12:45:44.596516 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 10 12:45:44.596516 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 10 12:45:44.596516 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application result = self._query(query) Jun 10 12:45:44.596516 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 10 12:45:44.596516 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application conn.query(q) Jun 10 12:45:44.596516 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 10 12:45:44.596516 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 10 12:45:44.596516 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 10 12:45:44.596516 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application result.read() Jun 10 12:45:44.596516 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 10 12:45:44.596516 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 10 12:45:44.597417 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 10 12:45:44.597417 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 10 12:45:44.597417 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 10 12:45:44.597417 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 10 12:45:44.597417 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 10 12:45:44.597417 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 10 12:45:44.597417 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 10 12:45:44.597417 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 10 12:45:44.597417 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [parameters: {'id': '8ddf939a3ce84055a13420921dee0166', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 10 12:45:44.597417 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 10 12:45:44.597417 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:44.597417 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 10 12:45:44.597417 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:44.597417 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:44.597417 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 10 12:45:44.598718 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 10 12:45:44.598718 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 10 12:45:44.598718 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 10 12:45:44.598718 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 10 12:45:44.598718 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:44.598718 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 10 12:45:44.598718 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:44.598718 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:44.598718 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:44.598718 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:44.598718 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:44.598718 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:44.598718 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:44.598718 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:44.598718 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:44.598718 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:44.598718 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:44.600884 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:44.600884 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 10 12:45:44.600884 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 10 12:45:44.600884 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:44.600884 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:44.600884 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 10 12:45:44.600884 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 10 12:45:44.600884 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 8ddf939a3ce84055a13420921dee0166. Jun 10 12:45:44.600884 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:44.600884 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 474/947] 10.222.0.27 () {60 vars in 1140 bytes} [Fri Jun 10 12:45:44 2022] POST /identity/v3/projects => generated 262 bytes in 219 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:44.602815 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35457ffa-a117-4dbb-b1a4-dd76ed24e36b None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:44.602815 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:44.700460 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35457ffa-a117-4dbb-b1a4-dd76ed24e36b None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:44.734573 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6fc120-0d49-46d4-bb5d-398f8e0d2b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:44.735376 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6fc120-0d49-46d4-bb5d-398f8e0d2b29 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:44.735660 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6fc120-0d49-46d4-bb5d-398f8e0d2b29 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:44.735966 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6fc120-0d49-46d4-bb5d-398f8e0d2b29 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:44.744288 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6fc120-0d49-46d4-bb5d-398f8e0d2b29 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:44.745779 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6fc120-0d49-46d4-bb5d-398f8e0d2b29 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:44.761262 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-ec6fc120-0d49-46d4-bb5d-398f8e0d2b29 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 10 12:45:44.761262 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:44.761262 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:44.761262 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:44.761262 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:44.761262 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:44.761262 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:44.761262 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:44.761262 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:44.761262 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:44.761262 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:44.761262 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:44.761262 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 10 12:45:44.761262 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 10 12:45:44.761262 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 10 12:45:44.762206 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 10 12:45:44.762206 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:44.762206 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:44.762206 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:44.762206 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:44.762206 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:44.762206 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:44.762206 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:44.762206 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:44.762206 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:44.762206 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:44.762206 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:44.762206 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:44.762206 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:44.762206 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:44.762206 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:44.762206 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:44.763426 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 10 12:45:44.763426 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 10 12:45:44.763426 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 10 12:45:44.763426 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 10 12:45:44.763426 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 10 12:45:44.763426 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 10 12:45:44.763426 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 10 12:45:44.763426 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:44.763426 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 474/948] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Jun 10 12:45:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:44.768686 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf9dac7-eddf-4f8b-bd47-65659347ee3a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:44.787312 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:44.878392 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf9dac7-eddf-4f8b-bd47-65659347ee3a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:44.913941 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ba4b5e-be46-4ae8-a3d9-2fe49fa028ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:44.914614 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ba4b5e-be46-4ae8-a3d9-2fe49fa028ff None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:44.914901 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ba4b5e-be46-4ae8-a3d9-2fe49fa028ff None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:44.915182 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ba4b5e-be46-4ae8-a3d9-2fe49fa028ff None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:44.916035 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ba4b5e-be46-4ae8-a3d9-2fe49fa028ff None admin] RBAC: Adding query filter params (name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:44.916349 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ba4b5e-be46-4ae8-a3d9-2fe49fa028ff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:44.917976 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ba4b5e-be46-4ae8-a3d9-2fe49fa028ff None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:44.939259 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 475/949] 10.222.0.27 () {56 vars in 1112 bytes} [Fri Jun 10 12:45:44 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:45.804843 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4d3799-f9db-4383-813c-f8a664970150 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:45.805489 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4d3799-f9db-4383-813c-f8a664970150 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:45.805720 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4d3799-f9db-4383-813c-f8a664970150 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:45.807491 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 475/950] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:45:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:45:45.812399 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f53bbe-e853-41f5-b0b0-ac2a2b61b601 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:45.812871 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f53bbe-e853-41f5-b0b0-ac2a2b61b601 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:45.813022 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f53bbe-e853-41f5-b0b0-ac2a2b61b601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:45.835273 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-82f53bbe-e853-41f5-b0b0-ac2a2b61b601 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:45:45.836946 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-82f53bbe-e853-41f5-b0b0-ac2a2b61b601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:45:45.907280 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 476/951] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:45:45 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 10 12:45:45.930132 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4d3799-f9db-4383-813c-f8a664970150 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:45.994477 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:46.040564 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4d3799-f9db-4383-813c-f8a664970150 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:46.076347 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f4abb0-3934-41a2-ae04-381dde81e1a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:46.077084 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f4abb0-3934-41a2-ae04-381dde81e1a0 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:46.077369 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f4abb0-3934-41a2-ae04-381dde81e1a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:46.077707 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f4abb0-3934-41a2-ae04-381dde81e1a0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:46.091671 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f4abb0-3934-41a2-ae04-381dde81e1a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:45:46.100222 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-64f4abb0-3934-41a2-ae04-381dde81e1a0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 10 12:45:46.100222 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:46.100222 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:46.100222 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:46.100222 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:46.100222 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:46.100222 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:46.100222 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:46.100222 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:46.100222 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:46.100222 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:46.100222 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:46.100222 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:45:46.100222 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:45:46.100222 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:45:46.100222 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:45:46.101324 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:45:46.101324 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:45:46.101324 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:45:46.101324 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:45:46.101324 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:46.101324 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:46.101324 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:46.101324 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:46.101324 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:46.101324 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:46.101324 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:46.101324 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:46.101324 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:46.101324 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:46.101324 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:46.101324 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:46.102255 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:46.102255 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:46.102255 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:46.102255 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:46.102255 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:45:46.102255 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:45:46.102255 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:45:46.102255 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:45:46.102255 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:45:46.102255 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:45:46.102255 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 10 12:45:46.102255 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:46.105358 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 476/952] 10.222.0.27 () {56 vars in 1120 bytes} [Fri Jun 10 12:45:45 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:46.111326 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f53bbe-e853-41f5-b0b0-ac2a2b61b601 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:46.121049 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:46.220931 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f53bbe-e853-41f5-b0b0-ac2a2b61b601 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:46.260877 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a721dd2-a68c-451a-9653-bdab2aabc1fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:46.261570 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a721dd2-a68c-451a-9653-bdab2aabc1fc None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:46.261850 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a721dd2-a68c-451a-9653-bdab2aabc1fc None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:46.262126 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a721dd2-a68c-451a-9653-bdab2aabc1fc None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:46.262884 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a721dd2-a68c-451a-9653-bdab2aabc1fc None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:46.263189 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a721dd2-a68c-451a-9653-bdab2aabc1fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:46.264609 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a721dd2-a68c-451a-9653-bdab2aabc1fc None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:46.273998 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 477/953] 10.222.0.27 () {56 vars in 1115 bytes} [Fri Jun 10 12:45:46 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:46.279522 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f4abb0-3934-41a2-ae04-381dde81e1a0 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:46.305281 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862b2aa7-dc7c-4c0b-abad-63d6278468e7 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:46.305629 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862b2aa7-dc7c-4c0b-abad-63d6278468e7 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:46.305851 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862b2aa7-dc7c-4c0b-abad-63d6278468e7 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:46.307838 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 478/954] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:45:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 10 12:45:46.312982 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ce8836-8030-4a7f-a66a-132de3565f83 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:46.313266 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ce8836-8030-4a7f-a66a-132de3565f83 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:46.313542 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ce8836-8030-4a7f-a66a-132de3565f83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:46.335515 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-09ce8836-8030-4a7f-a66a-132de3565f83 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:45:46.337034 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-09ce8836-8030-4a7f-a66a-132de3565f83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:45:46.404662 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f4abb0-3934-41a2-ae04-381dde81e1a0 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:46.414643 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:46.423002 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 479/955] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:45:46 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 10 12:45:46.433551 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:46.439613 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150ea929-3214-4339-b8bd-f77ca65aad1b None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:46.439943 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150ea929-3214-4339-b8bd-f77ca65aad1b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:46.440230 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150ea929-3214-4339-b8bd-f77ca65aad1b None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:46.443789 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 480/956] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:45:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 10 12:45:46.445748 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f50a9ad-41f6-490b-a890-ae024eb0919d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:46.446551 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f50a9ad-41f6-490b-a890-ae024eb0919d None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:46.446834 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f50a9ad-41f6-490b-a890-ae024eb0919d None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:46.447110 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f50a9ad-41f6-490b-a890-ae024eb0919d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:46.447312 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150ea929-3214-4339-b8bd-f77ca65aad1b None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:46.459854 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f50a9ad-41f6-490b-a890-ae024eb0919d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:46.461324 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f50a9ad-41f6-490b-a890-ae024eb0919d None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:46.479244 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-2f50a9ad-41f6-490b-a890-ae024eb0919d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:46.479244 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:46.479244 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:46.479244 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:46.479244 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:46.479244 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:46.479244 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:46.479244 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:46.479244 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:46.479244 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:46.479244 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:46.479244 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:46.479244 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 10 12:45:46.479244 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 10 12:45:46.479244 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 10 12:45:46.480228 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 10 12:45:46.480228 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:46.480228 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:46.480228 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:46.480228 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:46.480228 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:46.480228 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:46.480228 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:46.480228 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:46.480228 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:46.480228 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:46.480228 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:46.480228 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:46.480228 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:46.480228 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:46.480228 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:46.480228 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:46.481169 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 10 12:45:46.481169 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 10 12:45:46.481169 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:46.481169 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:46.481169 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 477/957] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:45:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:46.481724 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb7f757-414a-455f-961b-407a99d74e69 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:46.481825 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb7f757-414a-455f-961b-407a99d74e69 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:46.482362 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb7f757-414a-455f-961b-407a99d74e69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:46.499448 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-0eb7f757-414a-455f-961b-407a99d74e69 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:45:46.500636 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0eb7f757-414a-455f-961b-407a99d74e69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:45:46.557847 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150ea929-3214-4339-b8bd-f77ca65aad1b None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:46.571961 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 478/958] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:45:46 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 10 12:45:46.573035 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb7f757-414a-455f-961b-407a99d74e69 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:46.604148 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023e446f-a9c1-4d7b-82e2-7c281f48461d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:46.604884 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023e446f-a9c1-4d7b-82e2-7c281f48461d None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:46.605172 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023e446f-a9c1-4d7b-82e2-7c281f48461d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:46.605455 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023e446f-a9c1-4d7b-82e2-7c281f48461d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:46.606112 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023e446f-a9c1-4d7b-82e2-7c281f48461d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:46.607514 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023e446f-a9c1-4d7b-82e2-7c281f48461d None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:46.620617 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-023e446f-a9c1-4d7b-82e2-7c281f48461d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 10 12:45:46.620617 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:46.620617 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 10 12:45:46.620617 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 10 12:45:46.620617 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 10 12:45:46.620617 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 10 12:45:46.620617 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 10 12:45:46.620617 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 10 12:45:46.620617 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 10 12:45:46.620617 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:46.620617 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 10 12:45:46.620617 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:46.620617 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:46.620617 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:46.620617 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:46.620617 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:46.620617 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:46.621728 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:46.621728 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:46.621728 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:46.621728 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:46.621728 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:46.621728 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:46.621728 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 10 12:45:46.621728 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 10 12:45:46.621728 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 10 12:45:46.621728 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 10 12:45:46.621728 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:46.621728 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:46.621728 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:46.621728 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:46.621728 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:46.621728 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:46.622605 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:46.622605 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:46.622605 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:46.622605 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:46.622605 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:46.622605 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:46.622605 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:46.622605 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:46.622605 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:46.622605 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:46.622605 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 10 12:45:46.622605 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 10 12:45:46.622605 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 10 12:45:46.622605 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:46.622605 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 481/959] 10.222.0.27 () {56 vars in 1126 bytes} [Fri Jun 10 12:45:46 2022] GET /identity/v3/services/object-store => generated 93 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:46.623412 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023e446f-a9c1-4d7b-82e2-7c281f48461d None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:46.687254 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:46.693541 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb7f757-414a-455f-961b-407a99d74e69 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:46.701189 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:46.733109 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023e446f-a9c1-4d7b-82e2-7c281f48461d None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:46.733900 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cd0254-21bf-49bf-9160-e3c6c1204969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:46.734914 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cd0254-21bf-49bf-9160-e3c6c1204969 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:46.735356 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cd0254-21bf-49bf-9160-e3c6c1204969 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:46.735786 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cd0254-21bf-49bf-9160-e3c6c1204969 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:46.737126 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cd0254-21bf-49bf-9160-e3c6c1204969 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:46.737748 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cd0254-21bf-49bf-9160-e3c6c1204969 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:46.739743 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cd0254-21bf-49bf-9160-e3c6c1204969 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:46.753302 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 479/960] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Jun 10 12:45:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:46.754122 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cd0254-21bf-49bf-9160-e3c6c1204969 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:46.773085 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e4b1a7-1c92-4ed2-96af-996a125737df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:46.773884 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e4b1a7-1c92-4ed2-96af-996a125737df None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:46.774171 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e4b1a7-1c92-4ed2-96af-996a125737df None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:46.774454 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e4b1a7-1c92-4ed2-96af-996a125737df None admin] PATH_INFO: `/v3/services/compute` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:46.775113 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e4b1a7-1c92-4ed2-96af-996a125737df None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:46.776499 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e4b1a7-1c92-4ed2-96af-996a125737df None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:46.789060 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-a5e4b1a7-1c92-4ed2-96af-996a125737df None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 10 12:45:46.789060 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:46.789060 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 10 12:45:46.789060 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 10 12:45:46.789060 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 10 12:45:46.789060 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 10 12:45:46.789060 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 10 12:45:46.789060 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 10 12:45:46.789060 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 10 12:45:46.789060 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:46.789060 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 10 12:45:46.789060 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:46.789060 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:46.789060 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:46.789060 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:46.789060 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:46.789060 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:46.790063 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:46.790063 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:46.790063 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:46.790063 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:46.790063 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:46.790063 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:46.790063 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 10 12:45:46.790063 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 10 12:45:46.790063 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 10 12:45:46.790063 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 10 12:45:46.790063 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:46.790063 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:46.790063 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:46.790063 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:46.790063 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:46.790063 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:46.790908 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:46.790908 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:46.790908 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:46.790908 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:46.790908 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:46.790908 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:46.790908 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:46.790908 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:46.790908 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:46.790908 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:46.790908 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 10 12:45:46.790908 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 10 12:45:46.790908 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 10 12:45:46.790908 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:46.790908 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 482/961] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Jun 10 12:45:46 2022] GET /identity/v3/services/compute => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:46.791722 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e4b1a7-1c92-4ed2-96af-996a125737df None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:46.877725 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cd0254-21bf-49bf-9160-e3c6c1204969 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:46.900895 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e4b1a7-1c92-4ed2-96af-996a125737df None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:46.916783 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52692bc-204a-42c5-a40a-2ae4a54f5238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:46.917678 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52692bc-204a-42c5-a40a-2ae4a54f5238 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:46.918114 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52692bc-204a-42c5-a40a-2ae4a54f5238 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:46.918540 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52692bc-204a-42c5-a40a-2ae4a54f5238 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:46.919418 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52692bc-204a-42c5-a40a-2ae4a54f5238 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:46.919902 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52692bc-204a-42c5-a40a-2ae4a54f5238 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:46.921431 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52692bc-204a-42c5-a40a-2ae4a54f5238 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:46.925525 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:46.927557 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 480/962] 10.222.0.27 () {56 vars in 1122 bytes} [Fri Jun 10 12:45:46 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:46.928549 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52692bc-204a-42c5-a40a-2ae4a54f5238 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:46.940322 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d9fe7f-2cad-4c97-846d-0d61df9d0515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:46.941335 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d9fe7f-2cad-4c97-846d-0d61df9d0515 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:46.941638 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d9fe7f-2cad-4c97-846d-0d61df9d0515 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:46.941909 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d9fe7f-2cad-4c97-846d-0d61df9d0515 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:46.949724 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d9fe7f-2cad-4c97-846d-0d61df9d0515 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:46.951315 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d9fe7f-2cad-4c97-846d-0d61df9d0515 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:46.963663 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:46.967510 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-62d9fe7f-2cad-4c97-846d-0d61df9d0515 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 10 12:45:46.967510 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:46.967510 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:46.967510 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:46.967510 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:46.967510 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:46.967510 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:46.967510 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:46.967510 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:46.967510 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:46.967510 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:46.967510 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:46.967510 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 10 12:45:46.967510 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 10 12:45:46.967510 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 10 12:45:46.967510 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 10 12:45:46.968503 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:46.968503 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:46.968503 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 10 12:45:46.968503 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 10 12:45:46.968503 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 10 12:45:46.968503 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 10 12:45:46.968503 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:46.968503 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:46.968503 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:46.968503 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:46.968503 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:46.968503 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:46.968503 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:46.968503 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:46.968503 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:46.968503 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:46.968503 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:46.968503 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:46.969558 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:46.969558 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:46.969558 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 10 12:45:46.969558 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 10 12:45:46.969558 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 10 12:45:46.969558 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 10 12:45:46.969558 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 10 12:45:46.969558 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 10 12:45:46.969558 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 10 12:45:46.969558 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:46.969558 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 483/963] 10.222.0.27 () {56 vars in 1117 bytes} [Fri Jun 10 12:45:46 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:46.969558 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d9fe7f-2cad-4c97-846d-0d61df9d0515 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:47.040227 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52692bc-204a-42c5-a40a-2ae4a54f5238 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:47.074609 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3c55db-8386-4448-b46c-43537cc5e396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:47.075274 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3c55db-8386-4448-b46c-43537cc5e396 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:47.075558 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3c55db-8386-4448-b46c-43537cc5e396 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:47.075833 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3c55db-8386-4448-b46c-43537cc5e396 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:47.076572 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3c55db-8386-4448-b46c-43537cc5e396 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:47.076876 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3c55db-8386-4448-b46c-43537cc5e396 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:47.078268 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3c55db-8386-4448-b46c-43537cc5e396 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:47.088102 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 481/964] 10.222.0.27 () {56 vars in 1112 bytes} [Fri Jun 10 12:45:46 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:47.088102 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3c55db-8386-4448-b46c-43537cc5e396 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:47.102733 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d9fe7f-2cad-4c97-846d-0d61df9d0515 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:47.141110 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b83c97-3053-48f2-87cb-acb32db061e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:47.142180 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b83c97-3053-48f2-87cb-acb32db061e7 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:47.142688 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b83c97-3053-48f2-87cb-acb32db061e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:47.143196 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b83c97-3053-48f2-87cb-acb32db061e7 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:47.144201 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b83c97-3053-48f2-87cb-acb32db061e7 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:47.144748 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b83c97-3053-48f2-87cb-acb32db061e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:47.146387 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b83c97-3053-48f2-87cb-acb32db061e7 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:47.153323 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 484/965] 10.222.0.27 () {56 vars in 1122 bytes} [Fri Jun 10 12:45:46 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:47.154670 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b83c97-3053-48f2-87cb-acb32db061e7 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:47.188919 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:47.196476 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3c55db-8386-4448-b46c-43537cc5e396 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:47.209966 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:47.231174 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82680c50-8a70-4de8-8ce8-6555dd0dc677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:47.231841 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82680c50-8a70-4de8-8ce8-6555dd0dc677 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:47.232148 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82680c50-8a70-4de8-8ce8-6555dd0dc677 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:47.232430 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82680c50-8a70-4de8-8ce8-6555dd0dc677 None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:47.233309 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82680c50-8a70-4de8-8ce8-6555dd0dc677 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:47.233623 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82680c50-8a70-4de8-8ce8-6555dd0dc677 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:47.235056 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82680c50-8a70-4de8-8ce8-6555dd0dc677 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:47.253298 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 482/966] 10.222.0.27 () {56 vars in 1149 bytes} [Fri Jun 10 12:45:47 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:47.254063 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82680c50-8a70-4de8-8ce8-6555dd0dc677 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:47.261249 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b83c97-3053-48f2-87cb-acb32db061e7 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:47.294479 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63ece32-b0ef-484d-92de-c3312f1f335b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:47.295143 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63ece32-b0ef-484d-92de-c3312f1f335b None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:47.295430 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63ece32-b0ef-484d-92de-c3312f1f335b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:47.295710 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63ece32-b0ef-484d-92de-c3312f1f335b None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:47.296456 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63ece32-b0ef-484d-92de-c3312f1f335b None admin] RBAC: Adding query filter params (type=compute) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:47.296766 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63ece32-b0ef-484d-92de-c3312f1f335b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:47.298119 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63ece32-b0ef-484d-92de-c3312f1f335b None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:47.303146 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 485/967] 10.222.0.27 () {56 vars in 1112 bytes} [Fri Jun 10 12:45:47 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:47.304439 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23c2a05-782f-4828-9eaf-2b3a54f1defc None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:47.304725 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23c2a05-782f-4828-9eaf-2b3a54f1defc None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:47.305008 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23c2a05-782f-4828-9eaf-2b3a54f1defc None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:47.306267 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 486/968] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:45:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:45:47.306995 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23c2a05-782f-4828-9eaf-2b3a54f1defc None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:47.367143 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82680c50-8a70-4de8-8ce8-6555dd0dc677 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:47.406214 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181fa81f-5089-4a28-acf2-42e507ddb541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:47.407228 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181fa81f-5089-4a28-acf2-42e507ddb541 None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:47.407723 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181fa81f-5089-4a28-acf2-42e507ddb541 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:47.408222 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181fa81f-5089-4a28-acf2-42e507ddb541 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:47.409185 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181fa81f-5089-4a28-acf2-42e507ddb541 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:47.411142 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181fa81f-5089-4a28-acf2-42e507ddb541 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:47.423837 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23c2a05-782f-4828-9eaf-2b3a54f1defc None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:47.438395 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:47.460655 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7d80a0-a8f4-41d0-8f17-64721fc4b64a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:47.461308 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7d80a0-a8f4-41d0-8f17-64721fc4b64a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:47.461603 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7d80a0-a8f4-41d0-8f17-64721fc4b64a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:47.461923 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7d80a0-a8f4-41d0-8f17-64721fc4b64a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:47.498743 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 483/969] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Jun 10 12:45:47 2022] POST /identity/v3/endpoints => generated 349 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 12:45:47.500094 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d608635f-3f74-4426-a9fd-d66c330f415b None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:47.500320 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d608635f-3f74-4426-a9fd-d66c330f415b None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:47.500611 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d608635f-3f74-4426-a9fd-d66c330f415b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:47.518681 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-d608635f-3f74-4426-a9fd-d66c330f415b None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:45:47.519773 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d608635f-3f74-4426-a9fd-d66c330f415b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:45:47.523920 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:47.560593 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7d80a0-a8f4-41d0-8f17-64721fc4b64a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:47.563573 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7d80a0-a8f4-41d0-8f17-64721fc4b64a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:47.610482 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 484/970] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:45:47 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 10 12:45:47.635586 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d608635f-3f74-4426-a9fd-d66c330f415b None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:47.700859 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 487/971] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Jun 10 12:45:47 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 10 12:45:47.701832 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7d80a0-a8f4-41d0-8f17-64721fc4b64a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:47.718643 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:47.765175 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d608635f-3f74-4426-a9fd-d66c330f415b None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:47.813565 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fd0b69-19d2-4fc4-8064-75be7593ed7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:47.813809 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7d80a0-a8f4-41d0-8f17-64721fc4b64a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:47.814749 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd0b69-19d2-4fc4-8064-75be7593ed7a None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:47.815507 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd0b69-19d2-4fc4-8064-75be7593ed7a None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:47.816056 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd0b69-19d2-4fc4-8064-75be7593ed7a None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:47.817871 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fd0b69-19d2-4fc4-8064-75be7593ed7a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:47.819668 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fd0b69-19d2-4fc4-8064-75be7593ed7a None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:47.821225 dv-cinder-master-3-smb devstack@keystone.service[80238]: WARNING keystone.server.flask.common [None req-91fd0b69-19d2-4fc4-8064-75be7593ed7a None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 10 12:45:47.830566 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:47.856369 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b49e2c6-3674-4d55-8079-5e6786c572ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:47.857233 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b49e2c6-3674-4d55-8079-5e6786c572ef None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:47.857668 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b49e2c6-3674-4d55-8079-5e6786c572ef None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:47.858003 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b49e2c6-3674-4d55-8079-5e6786c572ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:47.863674 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.sql.core [None req-91fd0b69-19d2-4fc4-8064-75be7593ed7a None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 10 12:45:47.863674 dv-cinder-master-3-smb devstack@keystone.service[80238]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 10 12:45:47.863674 dv-cinder-master-3-smb devstack@keystone.service[80238]: [parameters: {'user_id': '42d493b3567943a2b877e6e351804a86', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 10 12:45:47.863674 dv-cinder-master-3-smb devstack@keystone.service[80238]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80238) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 10 12:45:47.873405 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b49e2c6-3674-4d55-8079-5e6786c572ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:47.877537 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b49e2c6-3674-4d55-8079-5e6786c572ef None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:47.888210 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-91fd0b69-19d2-4fc4-8064-75be7593ed7a None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 10 12:45:47.888210 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:47.888210 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 10 12:45:47.888210 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 10 12:45:47.888210 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 10 12:45:47.888210 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 10 12:45:47.888210 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 10 12:45:47.888210 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application result = self._query(query) Jun 10 12:45:47.888210 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 10 12:45:47.888210 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application conn.query(q) Jun 10 12:45:47.888210 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 10 12:45:47.888210 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 10 12:45:47.888210 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 10 12:45:47.888210 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application result.read() Jun 10 12:45:47.888210 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 10 12:45:47.889494 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 10 12:45:47.889494 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 10 12:45:47.889494 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 10 12:45:47.889494 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 10 12:45:47.889494 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 10 12:45:47.889494 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 10 12:45:47.889494 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 10 12:45:47.889494 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 10 12:45:47.889494 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:47.889494 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 10 12:45:47.889494 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:47.889494 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:47.889494 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 10 12:45:47.889494 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 10 12:45:47.889494 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 10 12:45:47.889494 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 10 12:45:47.889494 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 10 12:45:47.889494 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application next(self.gen) Jun 10 12:45:47.890529 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 10 12:45:47.890529 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application yield resource Jun 10 12:45:47.890529 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 10 12:45:47.890529 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application next(self.gen) Jun 10 12:45:47.890529 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 10 12:45:47.890529 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application self.session.rollback() Jun 10 12:45:47.890529 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 10 12:45:47.890529 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application self.force_reraise() Jun 10 12:45:47.890529 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 10 12:45:47.890529 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise self.value Jun 10 12:45:47.890529 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 10 12:45:47.890529 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 10 12:45:47.890529 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 10 12:45:47.890529 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application session.commit() Jun 10 12:45:47.890529 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 10 12:45:47.890529 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 10 12:45:47.890529 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 10 12:45:47.891470 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._prepare_impl() Jun 10 12:45:47.891470 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 10 12:45:47.891470 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application self.session.flush() Jun 10 12:45:47.891470 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 10 12:45:47.891470 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._flush(objects) Jun 10 12:45:47.891470 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 10 12:45:47.891470 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 10 12:45:47.891470 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 10 12:45:47.891470 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application compat.raise_( Jun 10 12:45:47.891470 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 10 12:45:47.891470 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception Jun 10 12:45:47.891470 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 10 12:45:47.891470 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application flush_context.execute() Jun 10 12:45:47.891470 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 10 12:45:47.891470 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rec.execute(self) Jun 10 12:45:47.891470 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 10 12:45:47.891470 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 10 12:45:47.892461 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 10 12:45:47.892461 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 10 12:45:47.892461 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 10 12:45:47.892461 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 10 12:45:47.892461 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 10 12:45:47.892461 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 10 12:45:47.892461 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 10 12:45:47.892461 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 10 12:45:47.892461 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 10 12:45:47.892461 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 10 12:45:47.892461 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 10 12:45:47.892461 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 10 12:45:47.892461 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 10 12:45:47.892461 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 10 12:45:47.892461 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 10 12:45:47.892461 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception Jun 10 12:45:47.893355 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 10 12:45:47.893355 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 10 12:45:47.893355 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 10 12:45:47.893355 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 10 12:45:47.893355 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 10 12:45:47.893355 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application result = self._query(query) Jun 10 12:45:47.893355 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 10 12:45:47.893355 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application conn.query(q) Jun 10 12:45:47.893355 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 10 12:45:47.893355 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 10 12:45:47.893355 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 10 12:45:47.893355 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application result.read() Jun 10 12:45:47.893355 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 10 12:45:47.893355 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 10 12:45:47.893355 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 10 12:45:47.893355 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 10 12:45:47.894209 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 10 12:45:47.894209 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 10 12:45:47.894209 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 10 12:45:47.894209 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 10 12:45:47.894209 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 10 12:45:47.894209 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 10 12:45:47.894209 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [parameters: {'user_id': '42d493b3567943a2b877e6e351804a86', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 10 12:45:47.894209 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 10 12:45:47.894209 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:47.894209 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 10 12:45:47.894209 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:47.894209 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:47.894209 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:47.894209 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:47.894209 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:47.895029 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:47.895029 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:47.895029 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:47.895029 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:47.895029 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:47.895029 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:47.895029 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:47.895029 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 10 12:45:47.895029 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 10 12:45:47.895029 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:47.895029 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:47.895029 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 10 12:45:47.895029 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 10 12:45:47.895029 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 10 12:45:47.895029 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 10 12:45:47.895029 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 10 12:45:47.895029 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 10 12:45:47.898088 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 10 12:45:47.898088 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 10 12:45:47.898088 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 10 12:45:47.898088 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 10 12:45:47.898088 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 10 12:45:47.898088 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:47.900228 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 485/972] 10.222.0.27 () {60 vars in 1131 bytes} [Fri Jun 10 12:45:47 2022] POST /identity/v3/users => generated 160 bytes in 265 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 10 12:45:47.901999 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fd0b69-19d2-4fc4-8064-75be7593ed7a None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:47.907189 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-1b49e2c6-3674-4d55-8079-5e6786c572ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:47.907189 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:47.907189 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:47.907189 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:47.907189 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:47.907189 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:47.907189 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:47.907189 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:47.907189 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:47.907189 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:47.907189 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:47.907189 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:47.907189 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 10 12:45:47.907189 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 10 12:45:47.907189 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 10 12:45:47.908470 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 10 12:45:47.908470 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:47.908470 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:47.908470 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:47.908470 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:47.908470 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:47.908470 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:47.908470 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:47.908470 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:47.908470 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:47.908470 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:47.908470 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:47.908470 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:47.908470 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:47.908470 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:47.908470 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:47.908470 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:47.909401 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 10 12:45:47.909401 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 10 12:45:47.909401 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:47.909401 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:47.914083 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 488/973] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:45:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 10 12:45:47.914962 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b49e2c6-3674-4d55-8079-5e6786c572ef None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:47.979355 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:48.018539 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b49e2c6-3674-4d55-8079-5e6786c572ef None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:48.023352 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fd0b69-19d2-4fc4-8064-75be7593ed7a None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:48.054797 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f2d7d1-0c13-4dfe-84e2-47b8673d6527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:48.055472 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f2d7d1-0c13-4dfe-84e2-47b8673d6527 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:48.055762 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f2d7d1-0c13-4dfe-84e2-47b8673d6527 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:48.056080 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f2d7d1-0c13-4dfe-84e2-47b8673d6527 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:48.056816 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f2d7d1-0c13-4dfe-84e2-47b8673d6527 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:48.057127 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f2d7d1-0c13-4dfe-84e2-47b8673d6527 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:48.058501 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f2d7d1-0c13-4dfe-84e2-47b8673d6527 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:48.064457 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9940c4b-74cc-445b-bb14-9da16aaeb509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:48.065512 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9940c4b-74cc-445b-bb14-9da16aaeb509 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:48.065996 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9940c4b-74cc-445b-bb14-9da16aaeb509 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:48.066504 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9940c4b-74cc-445b-bb14-9da16aaeb509 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:48.071142 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 489/974] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Jun 10 12:45:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:48.076282 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9940c4b-74cc-445b-bb14-9da16aaeb509 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:48.076722 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f2d7d1-0c13-4dfe-84e2-47b8673d6527 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:48.078586 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9940c4b-74cc-445b-bb14-9da16aaeb509 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:48.094577 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-e9940c4b-74cc-445b-bb14-9da16aaeb509 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 10 12:45:48.094577 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:48.094577 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:48.094577 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:48.094577 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:48.094577 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:48.094577 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:48.094577 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:48.094577 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:48.094577 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:48.094577 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:48.094577 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:48.094577 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 10 12:45:48.094577 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 10 12:45:48.094577 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 10 12:45:48.094577 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 10 12:45:48.095638 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:48.095638 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:48.095638 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 10 12:45:48.095638 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 10 12:45:48.095638 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 10 12:45:48.095638 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 10 12:45:48.095638 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:48.095638 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:48.095638 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:48.095638 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:48.095638 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:48.095638 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:48.095638 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:48.095638 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:48.095638 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:48.095638 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:48.095638 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:48.095638 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:48.097051 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:48.097051 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:48.097051 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 10 12:45:48.097051 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 10 12:45:48.097051 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 10 12:45:48.097051 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 10 12:45:48.097051 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 10 12:45:48.097051 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 10 12:45:48.097051 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 10 12:45:48.097051 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:48.099645 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 486/975] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Jun 10 12:45:47 2022] GET /identity/v3/users/cinder => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:48.105571 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9940c4b-74cc-445b-bb14-9da16aaeb509 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:48.167647 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:48.179306 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f2d7d1-0c13-4dfe-84e2-47b8673d6527 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:48.215038 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4d50ce-68d6-4c39-b133-6f3f9b10222c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:48.215840 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4d50ce-68d6-4c39-b133-6f3f9b10222c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:48.216162 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4d50ce-68d6-4c39-b133-6f3f9b10222c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:48.216400 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9940c4b-74cc-445b-bb14-9da16aaeb509 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:48.216526 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4d50ce-68d6-4c39-b133-6f3f9b10222c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:48.223375 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4d50ce-68d6-4c39-b133-6f3f9b10222c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:48.225122 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4d50ce-68d6-4c39-b133-6f3f9b10222c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:48.239903 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-fd4d50ce-68d6-4c39-b133-6f3f9b10222c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 10 12:45:48.239903 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:48.239903 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:48.239903 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:48.239903 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:48.239903 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:48.239903 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:48.239903 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:48.239903 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:48.239903 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:48.239903 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:48.239903 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:48.239903 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 10 12:45:48.239903 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 10 12:45:48.239903 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 10 12:45:48.240860 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 10 12:45:48.240860 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:48.240860 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:48.240860 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:48.240860 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:48.240860 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:48.240860 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:48.240860 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:48.240860 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:48.240860 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:48.240860 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:48.240860 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:48.240860 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:48.240860 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:48.240860 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:48.240860 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:48.240860 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:48.241804 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 10 12:45:48.241804 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 10 12:45:48.241804 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 10 12:45:48.241804 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 10 12:45:48.241804 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 10 12:45:48.241804 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 10 12:45:48.241804 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 10 12:45:48.241804 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:48.244211 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:48.245853 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 490/976] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Jun 10 12:45:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 10 12:45:48.246685 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4d50ce-68d6-4c39-b133-6f3f9b10222c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:48.258372 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f334f744-f187-42dc-9779-7484ec486bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:48.259408 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f334f744-f187-42dc-9779-7484ec486bef None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:48.259915 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f334f744-f187-42dc-9779-7484ec486bef None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:48.260717 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f334f744-f187-42dc-9779-7484ec486bef None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:48.261734 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f334f744-f187-42dc-9779-7484ec486bef None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:48.262300 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f334f744-f187-42dc-9779-7484ec486bef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:48.264041 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f334f744-f187-42dc-9779-7484ec486bef None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:48.311800 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 487/977] 10.222.0.27 () {56 vars in 1101 bytes} [Fri Jun 10 12:45:48 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 12:45:48.354762 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4d50ce-68d6-4c39-b133-6f3f9b10222c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:48.394514 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29525a4-dcc9-4939-91fe-35281f50ded4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:48.395190 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29525a4-dcc9-4939-91fe-35281f50ded4 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:48.395479 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29525a4-dcc9-4939-91fe-35281f50ded4 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:48.395759 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29525a4-dcc9-4939-91fe-35281f50ded4 None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:48.396645 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29525a4-dcc9-4939-91fe-35281f50ded4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:48.396962 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29525a4-dcc9-4939-91fe-35281f50ded4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:48.398454 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29525a4-dcc9-4939-91fe-35281f50ded4 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:48.412833 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 491/978] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Jun 10 12:45:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:48.421239 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f334f744-f187-42dc-9779-7484ec486bef None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:48.533133 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f334f744-f187-42dc-9779-7484ec486bef None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:48.548585 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:48.571356 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941c5a59-2b26-41b3-a3f5-fa4baf2f66ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:48.572366 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941c5a59-2b26-41b3-a3f5-fa4baf2f66ad None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:48.572853 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941c5a59-2b26-41b3-a3f5-fa4baf2f66ad None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:48.573303 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941c5a59-2b26-41b3-a3f5-fa4baf2f66ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:48.574394 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941c5a59-2b26-41b3-a3f5-fa4baf2f66ad None admin] RBAC: Adding query filter params (role.id=55ae4712a75e4f9bb542575264d55f64, scope.project.id=73ea5b23a2d34544ab493f4c5e4034b8, user.id=da66362070694e91bbe2cfbbd161b8a7) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:48.574946 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941c5a59-2b26-41b3-a3f5-fa4baf2f66ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:48.576636 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941c5a59-2b26-41b3-a3f5-fa4baf2f66ad None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:48.586452 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 488/979] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Jun 10 12:45:48 2022] GET /identity/v3/role_assignments?user.id=da66362070694e91bbe2cfbbd161b8a7&scope.project.id=73ea5b23a2d34544ab493f4c5e4034b8&role.id=55ae4712a75e4f9bb542575264d55f64 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:49.465558 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc47b19b-dbdd-4c5d-a282-737601917e64 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:49.466104 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc47b19b-dbdd-4c5d-a282-737601917e64 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:49.466104 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc47b19b-dbdd-4c5d-a282-737601917e64 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:49.467646 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 492/980] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:45:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:45:49.472640 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f507ee7-9a5b-48a3-b86a-bf652ff2656f None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:49.473026 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f507ee7-9a5b-48a3-b86a-bf652ff2656f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:49.473198 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f507ee7-9a5b-48a3-b86a-bf652ff2656f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:49.491025 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-9f507ee7-9a5b-48a3-b86a-bf652ff2656f None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:45:49.492553 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-9f507ee7-9a5b-48a3-b86a-bf652ff2656f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:45:49.559970 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 489/981] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:45:49 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 10 12:45:49.581833 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc47b19b-dbdd-4c5d-a282-737601917e64 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:49.628141 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:49.691463 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc47b19b-dbdd-4c5d-a282-737601917e64 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:49.708654 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02bfd72-0b26-4778-ad25-ab3fa97e0420 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:49.709034 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02bfd72-0b26-4778-ad25-ab3fa97e0420 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:49.709385 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02bfd72-0b26-4778-ad25-ab3fa97e0420 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:49.710835 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 490/982] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:45:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 10 12:45:49.715746 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f8a903-8829-4153-a241-d0dce524973e None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:49.716010 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f8a903-8829-4153-a241-d0dce524973e None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:49.716292 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f8a903-8829-4153-a241-d0dce524973e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:49.732489 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ca079c-3237-4a2d-8d5c-b9c5bf5110e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:49.733151 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ca079c-3237-4a2d-8d5c-b9c5bf5110e1 None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:49.733433 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ca079c-3237-4a2d-8d5c-b9c5bf5110e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:49.733550 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-21f8a903-8829-4153-a241-d0dce524973e None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:45:49.733704 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ca079c-3237-4a2d-8d5c-b9c5bf5110e1 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:49.734426 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ca079c-3237-4a2d-8d5c-b9c5bf5110e1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:49.734604 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-21f8a903-8829-4153-a241-d0dce524973e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:45:49.735768 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ca079c-3237-4a2d-8d5c-b9c5bf5110e1 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:49.804176 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 491/983] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:45:49 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 10 12:45:49.826025 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f8a903-8829-4153-a241-d0dce524973e None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:49.830685 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 493/984] 10.222.0.27 () {60 vars in 1141 bytes} [Fri Jun 10 12:45:49 2022] POST /identity/v3/services => generated 242 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 12:45:49.876297 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:49.941362 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f8a903-8829-4153-a241-d0dce524973e None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:49.984590 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc1a3a5-7845-4684-ae23-187993b63d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:49.985318 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc1a3a5-7845-4684-ae23-187993b63d7f None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:49.985596 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc1a3a5-7845-4684-ae23-187993b63d7f None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:49.985868 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc1a3a5-7845-4684-ae23-187993b63d7f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:49.986518 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc1a3a5-7845-4684-ae23-187993b63d7f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:49.987892 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc1a3a5-7845-4684-ae23-187993b63d7f None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:49.999934 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-edc1a3a5-7845-4684-ae23-187993b63d7f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 10 12:45:49.999934 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:49.999934 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 10 12:45:49.999934 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 10 12:45:49.999934 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 10 12:45:49.999934 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 10 12:45:49.999934 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 10 12:45:49.999934 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 10 12:45:49.999934 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 10 12:45:49.999934 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:49.999934 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 10 12:45:49.999934 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:49.999934 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:49.999934 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:49.999934 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:49.999934 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:49.999934 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:50.001162 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:50.001162 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:50.001162 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:50.001162 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:50.001162 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:50.001162 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:50.001162 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 10 12:45:50.001162 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 10 12:45:50.001162 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 10 12:45:50.001162 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 10 12:45:50.001162 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:50.001162 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:50.001162 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:50.001162 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:50.001162 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:50.001162 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:50.002235 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:50.002235 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:50.002235 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:50.002235 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:50.002235 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:50.002235 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:50.002235 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:50.002235 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:50.002235 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:50.002235 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:50.002235 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 10 12:45:50.002235 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 10 12:45:50.002235 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 10 12:45:50.002235 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:50.005980 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 492/985] 10.222.0.27 () {56 vars in 1126 bytes} [Fri Jun 10 12:45:49 2022] GET /identity/v3/services/object-store => generated 93 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 10 12:45:50.007117 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ca079c-3237-4a2d-8d5c-b9c5bf5110e1 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:50.108296 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ca079c-3237-4a2d-8d5c-b9c5bf5110e1 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:50.110642 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:50.141710 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfcb791-a467-403a-91d6-fe12dbbb59d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:50.142380 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfcb791-a467-403a-91d6-fe12dbbb59d4 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:50.142660 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfcb791-a467-403a-91d6-fe12dbbb59d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:50.142936 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfcb791-a467-403a-91d6-fe12dbbb59d4 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:50.143637 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfcb791-a467-403a-91d6-fe12dbbb59d4 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:50.143971 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfcb791-a467-403a-91d6-fe12dbbb59d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:50.145408 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfcb791-a467-403a-91d6-fe12dbbb59d4 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:50.152030 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 494/986] 10.222.0.27 () {56 vars in 1122 bytes} [Fri Jun 10 12:45:50 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:50.157664 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc1a3a5-7845-4684-ae23-187993b63d7f None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:50.256340 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:50.260687 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc1a3a5-7845-4684-ae23-187993b63d7f None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:50.296522 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fcffd3-3c85-4b6f-b861-5b144ccd9e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:50.297189 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fcffd3-3c85-4b6f-b861-5b144ccd9e9c None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:50.297468 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fcffd3-3c85-4b6f-b861-5b144ccd9e9c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:50.297746 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fcffd3-3c85-4b6f-b861-5b144ccd9e9c None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:50.298453 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fcffd3-3c85-4b6f-b861-5b144ccd9e9c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:50.298756 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fcffd3-3c85-4b6f-b861-5b144ccd9e9c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:50.300130 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fcffd3-3c85-4b6f-b861-5b144ccd9e9c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:50.305287 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 493/987] 10.222.0.27 () {56 vars in 1122 bytes} [Fri Jun 10 12:45:50 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:50.310739 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfcb791-a467-403a-91d6-fe12dbbb59d4 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:50.412398 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfcb791-a467-403a-91d6-fe12dbbb59d4 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:50.445273 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1856afe9-b2f7-478e-9daa-eaa99e73d13b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:50.445922 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1856afe9-b2f7-478e-9daa-eaa99e73d13b None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:50.446201 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1856afe9-b2f7-478e-9daa-eaa99e73d13b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:50.446471 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1856afe9-b2f7-478e-9daa-eaa99e73d13b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:50.447261 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1856afe9-b2f7-478e-9daa-eaa99e73d13b None admin] RBAC: Adding query filter params (interface=admin, service_id=8deed0a7aca34f638267dc35e549e9ad, region_id=RegionOne) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:50.447570 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1856afe9-b2f7-478e-9daa-eaa99e73d13b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:50.449011 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1856afe9-b2f7-478e-9daa-eaa99e73d13b None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:50.459186 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 495/988] 10.222.0.27 () {56 vars in 1249 bytes} [Fri Jun 10 12:45:50 2022] GET /identity/v3/endpoints?service_id=8deed0a7aca34f638267dc35e549e9ad&interface=admin®ion_id=RegionOne => generated 193 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:50.460345 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fcffd3-3c85-4b6f-b861-5b144ccd9e9c None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:50.557818 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fcffd3-3c85-4b6f-b861-5b144ccd9e9c None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:50.592201 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5161235d-b1dd-4169-bc2b-7725b50fb656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:50.592862 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5161235d-b1dd-4169-bc2b-7725b50fb656 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:50.593144 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5161235d-b1dd-4169-bc2b-7725b50fb656 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:50.593414 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5161235d-b1dd-4169-bc2b-7725b50fb656 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:50.594083 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5161235d-b1dd-4169-bc2b-7725b50fb656 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:50.595452 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5161235d-b1dd-4169-bc2b-7725b50fb656 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:50.607891 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 494/989] 10.222.0.27 () {56 vars in 1087 bytes} [Fri Jun 10 12:45:50 2022] GET /identity/v3/services => generated 1727 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 12:45:50.741800 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f26b052-d787-4dea-b1a1-38ff7ea851d4 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:50.742349 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f26b052-d787-4dea-b1a1-38ff7ea851d4 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:50.742349 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f26b052-d787-4dea-b1a1-38ff7ea851d4 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:50.743973 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 496/990] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:45:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:45:50.749532 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1c0c10-3fb5-446d-b746-27321d137c08 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:50.749825 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1c0c10-3fb5-446d-b746-27321d137c08 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:50.750103 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1c0c10-3fb5-446d-b746-27321d137c08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:50.767878 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-2f1c0c10-3fb5-446d-b746-27321d137c08 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:45:50.769092 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-2f1c0c10-3fb5-446d-b746-27321d137c08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:45:50.791379 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:50.839461 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 495/991] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:45:50 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 10 12:45:50.861436 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f26b052-d787-4dea-b1a1-38ff7ea851d4 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:50.895111 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:50.970668 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f26b052-d787-4dea-b1a1-38ff7ea851d4 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:51.005848 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f889c61-7386-47d3-b1bf-c555873d0822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:51.006639 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f889c61-7386-47d3-b1bf-c555873d0822 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:51.006923 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f889c61-7386-47d3-b1bf-c555873d0822 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:51.007198 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f889c61-7386-47d3-b1bf-c555873d0822 None admin] PATH_INFO: `/v3/services/image` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:51.007875 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f889c61-7386-47d3-b1bf-c555873d0822 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:51.009255 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f889c61-7386-47d3-b1bf-c555873d0822 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:51.021059 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-4f889c61-7386-47d3-b1bf-c555873d0822 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 10 12:45:51.021059 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:51.021059 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 10 12:45:51.021059 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 10 12:45:51.021059 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 10 12:45:51.021059 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 10 12:45:51.021059 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 10 12:45:51.021059 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 10 12:45:51.021059 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 10 12:45:51.021059 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:51.021059 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 10 12:45:51.021059 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:51.021059 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:51.021059 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:51.021059 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:51.021059 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:51.021059 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:51.022064 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:51.022064 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:51.022064 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:51.022064 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:51.022064 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:51.022064 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:51.022064 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 10 12:45:51.022064 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 10 12:45:51.022064 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 10 12:45:51.022064 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 10 12:45:51.022064 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:51.022064 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:51.022064 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:51.022064 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:51.022064 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:51.022064 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:51.022905 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:51.022905 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:51.022905 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:51.022905 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:51.022905 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:51.022905 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:51.022905 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:51.022905 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:51.022905 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:51.022905 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:51.022905 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 10 12:45:51.022905 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 10 12:45:51.022905 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 10 12:45:51.022905 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:51.022905 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 497/992] 10.222.0.27 () {56 vars in 1105 bytes} [Fri Jun 10 12:45:50 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:51.031019 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1c0c10-3fb5-446d-b746-27321d137c08 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:51.134320 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1c0c10-3fb5-446d-b746-27321d137c08 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:51.169554 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105d4ec9-a230-4dd4-8bbe-b28459bc008f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:51.170290 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105d4ec9-a230-4dd4-8bbe-b28459bc008f None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:51.170578 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105d4ec9-a230-4dd4-8bbe-b28459bc008f None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:51.170860 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105d4ec9-a230-4dd4-8bbe-b28459bc008f None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:51.171573 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105d4ec9-a230-4dd4-8bbe-b28459bc008f None admin] RBAC: Adding query filter params (name=image) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:51.171905 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105d4ec9-a230-4dd4-8bbe-b28459bc008f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:51.173358 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105d4ec9-a230-4dd4-8bbe-b28459bc008f None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:51.179032 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 496/993] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:45:51 2022] GET /identity/v3/services?name=image => generated 122 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:51.184406 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f889c61-7386-47d3-b1bf-c555873d0822 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:51.278517 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:51.284829 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f889c61-7386-47d3-b1bf-c555873d0822 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:51.320156 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bc3233-4aef-42be-83aa-eb9b013700ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:51.320831 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bc3233-4aef-42be-83aa-eb9b013700ba None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:51.321119 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bc3233-4aef-42be-83aa-eb9b013700ba None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:51.321399 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bc3233-4aef-42be-83aa-eb9b013700ba None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:51.322111 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bc3233-4aef-42be-83aa-eb9b013700ba None admin] RBAC: Adding query filter params (type=image) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:51.322424 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bc3233-4aef-42be-83aa-eb9b013700ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:51.324237 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bc3233-4aef-42be-83aa-eb9b013700ba None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:51.329760 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 498/994] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:45:51 2022] GET /identity/v3/services?type=image => generated 122 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 12:45:51.989030 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc6bebb-ce54-4c3f-8265-9e935c36c299 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:51.989536 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc6bebb-ce54-4c3f-8265-9e935c36c299 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:51.989650 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc6bebb-ce54-4c3f-8265-9e935c36c299 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:51.991102 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 497/995] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:45:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:45:51.996105 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507db32f-bec2-4bb8-889c-d0db12baf37e None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:51.996498 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507db32f-bec2-4bb8-889c-d0db12baf37e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:51.996670 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507db32f-bec2-4bb8-889c-d0db12baf37e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:52.013798 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-507db32f-bec2-4bb8-889c-d0db12baf37e None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:45:52.014987 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-507db32f-bec2-4bb8-889c-d0db12baf37e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:45:52.082979 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 499/996] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:45:51 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 10 12:45:52.106102 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc6bebb-ce54-4c3f-8265-9e935c36c299 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:52.129528 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:52.211848 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc6bebb-ce54-4c3f-8265-9e935c36c299 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:52.245200 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80856a3-8a08-4cef-81c3-f2f90cfdc89d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:52.245990 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80856a3-8a08-4cef-81c3-f2f90cfdc89d None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:52.246276 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80856a3-8a08-4cef-81c3-f2f90cfdc89d None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:52.246554 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80856a3-8a08-4cef-81c3-f2f90cfdc89d None admin] PATH_INFO: `/v3/services/compute` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:52.247203 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80856a3-8a08-4cef-81c3-f2f90cfdc89d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:52.248600 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80856a3-8a08-4cef-81c3-f2f90cfdc89d None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:52.260664 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-e80856a3-8a08-4cef-81c3-f2f90cfdc89d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 10 12:45:52.260664 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:52.260664 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 10 12:45:52.260664 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 10 12:45:52.260664 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 10 12:45:52.260664 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 10 12:45:52.260664 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 10 12:45:52.260664 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 10 12:45:52.260664 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 10 12:45:52.260664 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:52.260664 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 10 12:45:52.260664 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:52.260664 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:52.260664 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:52.260664 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:52.260664 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:52.260664 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:52.261645 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:52.261645 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:52.261645 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:52.261645 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:52.261645 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:52.261645 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:52.261645 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 10 12:45:52.261645 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 10 12:45:52.261645 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 10 12:45:52.261645 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 10 12:45:52.261645 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:52.261645 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:52.261645 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:52.261645 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:52.261645 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:52.261645 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:52.267298 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:52.267298 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:52.267298 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:52.267298 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:52.267298 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:52.267298 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:52.267298 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:52.267298 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:52.267298 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:52.267298 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:52.267298 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 10 12:45:52.267298 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 10 12:45:52.267298 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 10 12:45:52.267298 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:52.267298 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 498/997] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Jun 10 12:45:52 2022] GET /identity/v3/services/compute => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:52.269933 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507db32f-bec2-4bb8-889c-d0db12baf37e None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:52.359738 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:52.369871 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507db32f-bec2-4bb8-889c-d0db12baf37e None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:52.406139 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f21658-916c-4fc0-9b16-db3863126af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:52.406794 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f21658-916c-4fc0-9b16-db3863126af7 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:52.407075 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f21658-916c-4fc0-9b16-db3863126af7 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:52.407346 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f21658-916c-4fc0-9b16-db3863126af7 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:52.408087 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f21658-916c-4fc0-9b16-db3863126af7 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:52.408391 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f21658-916c-4fc0-9b16-db3863126af7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:52.409752 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f21658-916c-4fc0-9b16-db3863126af7 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:52.420052 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 500/998] 10.222.0.27 () {56 vars in 1112 bytes} [Fri Jun 10 12:45:52 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:52.421005 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80856a3-8a08-4cef-81c3-f2f90cfdc89d None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:52.508819 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:52.524480 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80856a3-8a08-4cef-81c3-f2f90cfdc89d None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:52.560169 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf950bee-b233-4415-b623-391b06abe6da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:52.560832 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf950bee-b233-4415-b623-391b06abe6da None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:52.561111 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf950bee-b233-4415-b623-391b06abe6da None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:52.561386 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf950bee-b233-4415-b623-391b06abe6da None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:52.562087 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf950bee-b233-4415-b623-391b06abe6da None admin] RBAC: Adding query filter params (type=compute) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:52.562392 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf950bee-b233-4415-b623-391b06abe6da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:52.563746 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf950bee-b233-4415-b623-391b06abe6da None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:52.568855 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 499/999] 10.222.0.27 () {56 vars in 1112 bytes} [Fri Jun 10 12:45:52 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:52.574556 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f21658-916c-4fc0-9b16-db3863126af7 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:52.679124 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f21658-916c-4fc0-9b16-db3863126af7 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:52.714001 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef8f16c-8582-4c40-bd8d-b4e0f4287bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:52.714658 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef8f16c-8582-4c40-bd8d-b4e0f4287bac None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:52.714939 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef8f16c-8582-4c40-bd8d-b4e0f4287bac None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:52.715213 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef8f16c-8582-4c40-bd8d-b4e0f4287bac None admin] PATH_INFO: `/v3/endpoints` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:52.716028 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef8f16c-8582-4c40-bd8d-b4e0f4287bac None admin] RBAC: Adding query filter params (interface=public, service_id=6fcc76a2a825406ca5e86c9544937ea7, region_id=RegionOne) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:52.716342 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef8f16c-8582-4c40-bd8d-b4e0f4287bac None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:52.717693 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef8f16c-8582-4c40-bd8d-b4e0f4287bac None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:52.727527 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 501/1000] 10.222.0.27 () {56 vars in 1251 bytes} [Fri Jun 10 12:45:52 2022] GET /identity/v3/endpoints?service_id=6fcc76a2a825406ca5e86c9544937ea7&interface=public®ion_id=RegionOne => generated 194 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 12:45:52.729053 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf950bee-b233-4415-b623-391b06abe6da None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:52.732137 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f8f373-f5f5-481b-bb7d-7a5897b969f2 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:52.732426 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f8f373-f5f5-481b-bb7d-7a5897b969f2 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:52.732697 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f8f373-f5f5-481b-bb7d-7a5897b969f2 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:52.734195 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 502/1001] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:45:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 10 12:45:52.738779 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36001db0-5897-4c1b-b138-0b73c5bc1b15 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:52.739061 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36001db0-5897-4c1b-b138-0b73c5bc1b15 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:52.739340 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36001db0-5897-4c1b-b138-0b73c5bc1b15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:52.757801 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-36001db0-5897-4c1b-b138-0b73c5bc1b15 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:45:52.758979 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-36001db0-5897-4c1b-b138-0b73c5bc1b15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:45:52.775377 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:52.830295 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 503/1002] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:45:52 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 10 12:45:52.843935 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf950bee-b233-4415-b623-391b06abe6da None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:52.857873 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36001db0-5897-4c1b-b138-0b73c5bc1b15 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:52.891751 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c24ea2-6baa-4780-aa78-4bda67f5cf9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:52.892451 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c24ea2-6baa-4780-aa78-4bda67f5cf9d None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:52.892745 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c24ea2-6baa-4780-aa78-4bda67f5cf9d None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:52.893027 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c24ea2-6baa-4780-aa78-4bda67f5cf9d None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:52.893715 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c24ea2-6baa-4780-aa78-4bda67f5cf9d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:52.895087 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c24ea2-6baa-4780-aa78-4bda67f5cf9d None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:52.903899 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 500/1003] 10.222.0.27 () {56 vars in 1087 bytes} [Fri Jun 10 12:45:52 2022] GET /identity/v3/services => generated 1727 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 10 12:45:52.968359 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36001db0-5897-4c1b-b138-0b73c5bc1b15 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:53.005510 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c9823f-7d8b-435d-84e0-3799193f8393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:53.006339 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c9823f-7d8b-435d-84e0-3799193f8393 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:53.006627 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c9823f-7d8b-435d-84e0-3799193f8393 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:53.006906 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c9823f-7d8b-435d-84e0-3799193f8393 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:53.007574 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c9823f-7d8b-435d-84e0-3799193f8393 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:53.009006 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c9823f-7d8b-435d-84e0-3799193f8393 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:53.022403 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-00c9823f-7d8b-435d-84e0-3799193f8393 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 10 12:45:53.022403 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:53.022403 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 10 12:45:53.022403 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 10 12:45:53.022403 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 10 12:45:53.022403 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 10 12:45:53.022403 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 10 12:45:53.022403 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 10 12:45:53.022403 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 10 12:45:53.022403 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:53.022403 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 10 12:45:53.022403 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:53.022403 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:53.022403 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:53.022403 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:53.022403 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:53.022403 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:53.023512 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:53.023512 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:53.023512 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:53.023512 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:53.023512 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:53.023512 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:53.023512 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 10 12:45:53.023512 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 10 12:45:53.023512 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 10 12:45:53.023512 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 10 12:45:53.023512 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:53.023512 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:53.023512 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:53.023512 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:53.023512 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:53.023512 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:53.024882 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:53.024882 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:53.024882 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:53.024882 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:53.024882 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:53.024882 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:53.024882 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:53.024882 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:53.024882 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:53.024882 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:53.024882 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 10 12:45:53.024882 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 10 12:45:53.024882 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 10 12:45:53.024882 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:53.024882 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 504/1004] 10.222.0.27 () {56 vars in 1126 bytes} [Fri Jun 10 12:45:52 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 10 12:45:53.031980 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c24ea2-6baa-4780-aa78-4bda67f5cf9d None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:53.048325 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:53.140938 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c24ea2-6baa-4780-aa78-4bda67f5cf9d None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:53.177585 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2467c8e-1c94-42c6-9bc8-c266cb47eb55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:53.178251 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2467c8e-1c94-42c6-9bc8-c266cb47eb55 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:53.178535 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2467c8e-1c94-42c6-9bc8-c266cb47eb55 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:53.178853 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2467c8e-1c94-42c6-9bc8-c266cb47eb55 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:53.179566 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2467c8e-1c94-42c6-9bc8-c266cb47eb55 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:53.179885 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2467c8e-1c94-42c6-9bc8-c266cb47eb55 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:53.181261 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2467c8e-1c94-42c6-9bc8-c266cb47eb55 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:53.187980 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 501/1005] 10.222.0.27 () {56 vars in 1122 bytes} [Fri Jun 10 12:45:53 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:53.192402 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c9823f-7d8b-435d-84e0-3799193f8393 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:53.208859 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:53.294398 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c9823f-7d8b-435d-84e0-3799193f8393 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:53.327422 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e3509c-27b6-46a6-9463-3b7e39d3af2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:53.328111 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e3509c-27b6-46a6-9463-3b7e39d3af2d None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:53.328396 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e3509c-27b6-46a6-9463-3b7e39d3af2d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:53.328678 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e3509c-27b6-46a6-9463-3b7e39d3af2d None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:53.329380 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e3509c-27b6-46a6-9463-3b7e39d3af2d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:53.329686 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e3509c-27b6-46a6-9463-3b7e39d3af2d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:53.331030 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e3509c-27b6-46a6-9463-3b7e39d3af2d None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:53.336680 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 505/1006] 10.222.0.27 () {56 vars in 1122 bytes} [Fri Jun 10 12:45:53 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:53.341993 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2467c8e-1c94-42c6-9bc8-c266cb47eb55 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:53.423501 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:53.446611 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2467c8e-1c94-42c6-9bc8-c266cb47eb55 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:53.450841 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91bb022-9be2-4921-a662-e89fa097af6b None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:53.451203 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91bb022-9be2-4921-a662-e89fa097af6b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:53.451686 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91bb022-9be2-4921-a662-e89fa097af6b None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:53.453350 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 506/1007] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:45:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:45:53.458855 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d810eb3f-709d-4e9f-95fb-481f99d073aa None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:53.459233 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d810eb3f-709d-4e9f-95fb-481f99d073aa None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:53.459589 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d810eb3f-709d-4e9f-95fb-481f99d073aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:53.476238 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-d810eb3f-709d-4e9f-95fb-481f99d073aa None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:45:53.477524 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d810eb3f-709d-4e9f-95fb-481f99d073aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:45:53.484189 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b985b296-4946-4ddb-8ee7-e6bdb031916b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:53.484853 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b985b296-4946-4ddb-8ee7-e6bdb031916b None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:53.485141 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b985b296-4946-4ddb-8ee7-e6bdb031916b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:53.485407 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b985b296-4946-4ddb-8ee7-e6bdb031916b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:53.486153 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b985b296-4946-4ddb-8ee7-e6bdb031916b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:53.487518 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b985b296-4946-4ddb-8ee7-e6bdb031916b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:53.540268 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 502/1008] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Jun 10 12:45:53 2022] POST /identity/v3/endpoints => generated 325 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 10 12:45:53.560053 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 507/1009] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:45:53 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 10 12:45:53.576696 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b985b296-4946-4ddb-8ee7-e6bdb031916b None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:53.687569 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b985b296-4946-4ddb-8ee7-e6bdb031916b None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:53.725870 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c970b3dd-b951-4934-bbd5-fc4c22eff398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:53.726914 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c970b3dd-b951-4934-bbd5-fc4c22eff398 None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:53.727201 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c970b3dd-b951-4934-bbd5-fc4c22eff398 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:53.727474 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c970b3dd-b951-4934-bbd5-fc4c22eff398 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:53.728235 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c970b3dd-b951-4934-bbd5-fc4c22eff398 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:53.729600 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c970b3dd-b951-4934-bbd5-fc4c22eff398 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:53.733039 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:53.829351 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 503/1010] 10.222.0.27 () {60 vars in 1141 bytes} [Fri Jun 10 12:45:53 2022] POST /identity/v3/services => generated 242 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 12:45:55.063363 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac221cd3-06ff-4701-9ad8-97c739de431d None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:55.064049 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac221cd3-06ff-4701-9ad8-97c739de431d None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:55.064049 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac221cd3-06ff-4701-9ad8-97c739de431d None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:55.065367 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 508/1011] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:45:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:45:55.070915 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cfc278-6fee-44ed-b3be-57608afd21f8 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:55.071309 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cfc278-6fee-44ed-b3be-57608afd21f8 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:55.071485 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cfc278-6fee-44ed-b3be-57608afd21f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:55.088441 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-31cfc278-6fee-44ed-b3be-57608afd21f8 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:45:55.089631 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-31cfc278-6fee-44ed-b3be-57608afd21f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:45:55.161649 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 504/1012] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:45:55 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 10 12:45:55.184493 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac221cd3-06ff-4701-9ad8-97c739de431d None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:55.187426 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:55.286414 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac221cd3-06ff-4701-9ad8-97c739de431d None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:55.321628 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81de3d88-e85f-40df-b964-804a8a6072ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:55.322424 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81de3d88-e85f-40df-b964-804a8a6072ee None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:55.322712 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81de3d88-e85f-40df-b964-804a8a6072ee None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:55.322993 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81de3d88-e85f-40df-b964-804a8a6072ee None admin] PATH_INFO: `/v3/services/compute` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:55.323649 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81de3d88-e85f-40df-b964-804a8a6072ee None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:55.325045 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81de3d88-e85f-40df-b964-804a8a6072ee None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:55.336737 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-81de3d88-e85f-40df-b964-804a8a6072ee None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 10 12:45:55.336737 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:55.336737 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 10 12:45:55.336737 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 10 12:45:55.336737 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 10 12:45:55.336737 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 10 12:45:55.336737 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 10 12:45:55.336737 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 10 12:45:55.336737 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 10 12:45:55.336737 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:55.336737 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 10 12:45:55.336737 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:55.336737 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:55.336737 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:55.336737 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:55.336737 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:55.336737 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:55.337764 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:55.337764 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:55.337764 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:55.337764 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:55.337764 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:55.337764 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:55.337764 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 10 12:45:55.337764 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 10 12:45:55.337764 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 10 12:45:55.337764 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 10 12:45:55.337764 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:55.337764 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:55.337764 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:55.337764 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:55.337764 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:55.337764 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:55.338633 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:55.338633 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:55.338633 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:55.338633 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:55.338633 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:55.338633 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:55.338633 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:55.338633 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:55.338633 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:55.338633 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:55.338633 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 10 12:45:55.338633 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 10 12:45:55.338633 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 10 12:45:55.338633 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:55.345921 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 509/1013] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Jun 10 12:45:55 2022] GET /identity/v3/services/compute => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:55.346067 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cfc278-6fee-44ed-b3be-57608afd21f8 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:55.444630 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cfc278-6fee-44ed-b3be-57608afd21f8 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:55.467248 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:55.479847 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140088d7-458c-4769-abe4-40ee05a06c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:55.480547 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140088d7-458c-4769-abe4-40ee05a06c1e None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:55.480835 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140088d7-458c-4769-abe4-40ee05a06c1e None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:55.481113 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140088d7-458c-4769-abe4-40ee05a06c1e None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:55.481825 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140088d7-458c-4769-abe4-40ee05a06c1e None admin] RBAC: Adding query filter params (name=compute) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:55.482133 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140088d7-458c-4769-abe4-40ee05a06c1e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:55.483504 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140088d7-458c-4769-abe4-40ee05a06c1e None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:55.493516 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 505/1014] 10.222.0.27 () {56 vars in 1112 bytes} [Fri Jun 10 12:45:55 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:55.495024 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81de3d88-e85f-40df-b964-804a8a6072ee None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:55.560819 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:55.598144 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81de3d88-e85f-40df-b964-804a8a6072ee None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:55.637235 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e87dfc-a965-40fe-b049-1f769c84bf8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:55.637903 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e87dfc-a965-40fe-b049-1f769c84bf8a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:55.638195 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e87dfc-a965-40fe-b049-1f769c84bf8a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:55.638471 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e87dfc-a965-40fe-b049-1f769c84bf8a None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:55.639180 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e87dfc-a965-40fe-b049-1f769c84bf8a None admin] RBAC: Adding query filter params (type=compute) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:55.639488 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e87dfc-a965-40fe-b049-1f769c84bf8a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:55.640875 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e87dfc-a965-40fe-b049-1f769c84bf8a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:55.650332 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 510/1015] 10.222.0.27 () {56 vars in 1112 bytes} [Fri Jun 10 12:45:55 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:55.651877 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140088d7-458c-4769-abe4-40ee05a06c1e None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:55.658656 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796a6c34-a276-4b87-a2bf-ba6e75ab9c28 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:55.658943 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796a6c34-a276-4b87-a2bf-ba6e75ab9c28 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:55.659227 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796a6c34-a276-4b87-a2bf-ba6e75ab9c28 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:55.661050 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 511/1016] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:45:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 10 12:45:55.666514 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe15f7a2-6074-404c-b1ec-c1d014da14d9 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:55.666808 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe15f7a2-6074-404c-b1ec-c1d014da14d9 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:55.667089 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe15f7a2-6074-404c-b1ec-c1d014da14d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:55.684281 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-fe15f7a2-6074-404c-b1ec-c1d014da14d9 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:45:55.685478 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-fe15f7a2-6074-404c-b1ec-c1d014da14d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:45:55.754425 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 512/1017] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:45:55 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 10 12:45:55.773624 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140088d7-458c-4769-abe4-40ee05a06c1e None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:55.777675 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe15f7a2-6074-404c-b1ec-c1d014da14d9 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:55.812098 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e11fb67-c1d1-4247-9013-9e6a742b98bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:55.812745 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e11fb67-c1d1-4247-9013-9e6a742b98bd None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:55.813033 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e11fb67-c1d1-4247-9013-9e6a742b98bd None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:55.813310 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e11fb67-c1d1-4247-9013-9e6a742b98bd None admin] PATH_INFO: `/v3/endpoints` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:55.814051 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e11fb67-c1d1-4247-9013-9e6a742b98bd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:55.815417 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e11fb67-c1d1-4247-9013-9e6a742b98bd None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:55.840099 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:55.880147 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe15f7a2-6074-404c-b1ec-c1d014da14d9 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:55.904791 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 506/1018] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Jun 10 12:45:55 2022] POST /identity/v3/endpoints => generated 334 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 12:45:55.925898 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceed4c46-8969-496e-a96c-2f04b9864162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:55.926729 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceed4c46-8969-496e-a96c-2f04b9864162 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:55.927583 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceed4c46-8969-496e-a96c-2f04b9864162 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:55.928259 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceed4c46-8969-496e-a96c-2f04b9864162 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:55.930832 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceed4c46-8969-496e-a96c-2f04b9864162 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:55.932358 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceed4c46-8969-496e-a96c-2f04b9864162 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:55.934413 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 513/1019] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:45:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 12:45:55.940346 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e11fb67-c1d1-4247-9013-9e6a742b98bd None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:55.951229 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f0656c-f402-42dc-b592-af34e2e7913f None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:55.951557 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f0656c-f402-42dc-b592-af34e2e7913f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:55.951884 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f0656c-f402-42dc-b592-af34e2e7913f None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:55.953435 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 514/1020] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:45:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 10 12:45:55.960117 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da52dde-6814-48b9-ad71-83dfee356f4d None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:55.960774 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da52dde-6814-48b9-ad71-83dfee356f4d None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:55.961251 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da52dde-6814-48b9-ad71-83dfee356f4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:55.981123 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-4da52dde-6814-48b9-ad71-83dfee356f4d None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:45:55.982732 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-4da52dde-6814-48b9-ad71-83dfee356f4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:45:55.988488 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:56.050923 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e11fb67-c1d1-4247-9013-9e6a742b98bd None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:56.056922 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 515/1021] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:45:55 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 10 12:45:56.079519 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da52dde-6814-48b9-ad71-83dfee356f4d None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:56.090312 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3a4d07-899a-4d5c-b134-9f4dc2be3949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:56.090988 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3a4d07-899a-4d5c-b134-9f4dc2be3949 None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:56.092224 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3a4d07-899a-4d5c-b134-9f4dc2be3949 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:56.092224 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3a4d07-899a-4d5c-b134-9f4dc2be3949 None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:56.092524 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3a4d07-899a-4d5c-b134-9f4dc2be3949 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:56.093961 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3a4d07-899a-4d5c-b134-9f4dc2be3949 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:56.140478 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:56.184509 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da52dde-6814-48b9-ad71-83dfee356f4d None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:56.193772 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 507/1022] 10.222.0.27 () {60 vars in 1141 bytes} [Fri Jun 10 12:45:55 2022] POST /identity/v3/projects => generated 311 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 12:45:56.224591 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82efc240-23cd-47de-9685-a1c8678dc3e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:56.225411 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82efc240-23cd-47de-9685-a1c8678dc3e1 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:56.225691 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82efc240-23cd-47de-9685-a1c8678dc3e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:56.225964 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82efc240-23cd-47de-9685-a1c8678dc3e1 None admin] PATH_INFO: `/v3/services/image` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:56.226642 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82efc240-23cd-47de-9685-a1c8678dc3e1 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:56.228124 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82efc240-23cd-47de-9685-a1c8678dc3e1 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:56.241259 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-82efc240-23cd-47de-9685-a1c8678dc3e1 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 10 12:45:56.241259 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:56.241259 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 10 12:45:56.241259 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 10 12:45:56.241259 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 10 12:45:56.241259 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 10 12:45:56.241259 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 10 12:45:56.241259 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 10 12:45:56.241259 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 10 12:45:56.241259 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:56.241259 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 10 12:45:56.241259 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:56.241259 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:56.241259 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:56.241259 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:56.241259 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:56.241259 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:56.242538 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:56.242538 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:56.242538 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:56.242538 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:56.242538 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:56.242538 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:56.242538 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 10 12:45:56.242538 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 10 12:45:56.242538 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 10 12:45:56.242538 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 10 12:45:56.242538 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:56.242538 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:56.242538 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:56.242538 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:56.242538 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:56.242538 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:56.243587 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:56.243587 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:56.243587 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:56.243587 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:56.243587 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:56.243587 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:56.243587 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:56.243587 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:56.243587 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:56.243587 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:56.243587 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 10 12:45:56.243587 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 10 12:45:56.243587 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 10 12:45:56.243587 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:56.245992 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 516/1023] 10.222.0.27 () {56 vars in 1105 bytes} [Fri Jun 10 12:45:56 2022] GET /identity/v3/services/image => generated 86 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 10 12:45:56.252586 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3a4d07-899a-4d5c-b134-9f4dc2be3949 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:56.353753 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:56.360834 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3a4d07-899a-4d5c-b134-9f4dc2be3949 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:56.398654 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a12e50-fe88-4498-bb12-00e84e9fbe47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:56.399355 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a12e50-fe88-4498-bb12-00e84e9fbe47 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:56.399634 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a12e50-fe88-4498-bb12-00e84e9fbe47 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:56.399939 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a12e50-fe88-4498-bb12-00e84e9fbe47 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:56.400652 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a12e50-fe88-4498-bb12-00e84e9fbe47 None admin] RBAC: Adding query filter params (name=image) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:56.400949 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a12e50-fe88-4498-bb12-00e84e9fbe47 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:56.402321 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a12e50-fe88-4498-bb12-00e84e9fbe47 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:56.408619 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 508/1024] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:45:56 2022] GET /identity/v3/services?name=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:56.414499 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82efc240-23cd-47de-9685-a1c8678dc3e1 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:56.514483 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82efc240-23cd-47de-9685-a1c8678dc3e1 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:56.533643 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:56.550712 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600372b9-6021-40a7-a27e-9f9731885561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:56.551385 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600372b9-6021-40a7-a27e-9f9731885561 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:56.551670 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600372b9-6021-40a7-a27e-9f9731885561 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:56.552010 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600372b9-6021-40a7-a27e-9f9731885561 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:56.552734 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600372b9-6021-40a7-a27e-9f9731885561 None admin] RBAC: Adding query filter params (type=image) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:56.553291 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600372b9-6021-40a7-a27e-9f9731885561 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:56.554988 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600372b9-6021-40a7-a27e-9f9731885561 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:56.560324 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 517/1025] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:45:56 2022] GET /identity/v3/services?type=image => generated 350 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:56.566270 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a12e50-fe88-4498-bb12-00e84e9fbe47 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:56.666589 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a12e50-fe88-4498-bb12-00e84e9fbe47 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:56.701536 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f6e48e-0567-458e-be42-f38146c244a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:56.702200 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f6e48e-0567-458e-be42-f38146c244a3 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:56.702483 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f6e48e-0567-458e-be42-f38146c244a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:56.702760 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f6e48e-0567-458e-be42-f38146c244a3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:56.703551 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f6e48e-0567-458e-be42-f38146c244a3 None admin] RBAC: Adding query filter params (interface=public, service_id=6ca5297e3e36464082c31e58fb761711, region_id=RegionOne) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:56.703879 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f6e48e-0567-458e-be42-f38146c244a3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:56.705247 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f6e48e-0567-458e-be42-f38146c244a3 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:56.713817 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 509/1026] 10.222.0.27 () {56 vars in 1251 bytes} [Fri Jun 10 12:45:56 2022] GET /identity/v3/endpoints?service_id=6ca5297e3e36464082c31e58fb761711&interface=public®ion_id=RegionOne => generated 194 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:56.715942 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600372b9-6021-40a7-a27e-9f9731885561 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:56.815300 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600372b9-6021-40a7-a27e-9f9731885561 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:56.849505 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844a375e-0038-421c-bd09-cc658ee7ee91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:56.850176 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844a375e-0038-421c-bd09-cc658ee7ee91 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:56.850464 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844a375e-0038-421c-bd09-cc658ee7ee91 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:56.850742 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844a375e-0038-421c-bd09-cc658ee7ee91 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:56.851421 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844a375e-0038-421c-bd09-cc658ee7ee91 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:56.855305 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844a375e-0038-421c-bd09-cc658ee7ee91 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:56.867916 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 518/1027] 10.222.0.27 () {56 vars in 1087 bytes} [Fri Jun 10 12:45:56 2022] GET /identity/v3/services => generated 1957 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 12:45:58.157009 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2ffd59-c66a-4e1a-a59a-e88e30e4ad79 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:58.157516 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2ffd59-c66a-4e1a-a59a-e88e30e4ad79 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:58.157516 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2ffd59-c66a-4e1a-a59a-e88e30e4ad79 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:58.158983 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 510/1028] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:45:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:45:58.164902 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaffe4a-334a-4576-b728-84bf8ec20005 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:58.165349 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaffe4a-334a-4576-b728-84bf8ec20005 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:58.165568 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaffe4a-334a-4576-b728-84bf8ec20005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:58.182014 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-deaffe4a-334a-4576-b728-84bf8ec20005 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:45:58.183178 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-deaffe4a-334a-4576-b728-84bf8ec20005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:45:58.225920 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:58.250945 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 519/1029] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:45:58 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 10 12:45:58.272993 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2ffd59-c66a-4e1a-a59a-e88e30e4ad79 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:58.297925 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:58.376761 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2ffd59-c66a-4e1a-a59a-e88e30e4ad79 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:58.411664 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2803aa5-ac6e-41cf-9fe0-64c551460c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:58.412423 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2803aa5-ac6e-41cf-9fe0-64c551460c49 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:58.412703 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2803aa5-ac6e-41cf-9fe0-64c551460c49 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:58.412976 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2803aa5-ac6e-41cf-9fe0-64c551460c49 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:58.422887 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2803aa5-ac6e-41cf-9fe0-64c551460c49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:45:58.430912 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-f2803aa5-ac6e-41cf-9fe0-64c551460c49 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 10 12:45:58.430912 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:58.430912 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:58.430912 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:58.430912 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:58.430912 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:58.430912 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:58.430912 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:58.430912 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:58.430912 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:58.430912 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:58.430912 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:58.430912 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:45:58.430912 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:45:58.430912 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:45:58.430912 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:45:58.431853 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:45:58.431853 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:45:58.431853 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:45:58.431853 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:45:58.431853 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:58.431853 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:58.431853 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:58.431853 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:58.431853 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:58.431853 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:58.431853 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:58.431853 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:58.431853 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:58.431853 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:58.431853 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:58.431853 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:58.432983 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:58.432983 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:58.432983 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:58.432983 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:58.432983 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:45:58.432983 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:45:58.432983 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:45:58.432983 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:45:58.432983 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:45:58.432983 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:45:58.432983 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 10 12:45:58.432983 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:58.435938 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 511/1030] 10.222.0.27 () {56 vars in 1120 bytes} [Fri Jun 10 12:45:58 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:45:58.438276 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaffe4a-334a-4576-b728-84bf8ec20005 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:58.443774 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b72cee8-41f7-449d-b7b2-b482fac52ccf None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:58.444085 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b72cee8-41f7-449d-b7b2-b482fac52ccf None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:58.444364 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b72cee8-41f7-449d-b7b2-b482fac52ccf None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:58.445695 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 512/1031] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:45:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:45:58.450686 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deb711a-dafb-4e5a-ac7f-f1c7d24d2e08 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:58.450971 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deb711a-dafb-4e5a-ac7f-f1c7d24d2e08 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:58.451251 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deb711a-dafb-4e5a-ac7f-f1c7d24d2e08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:58.468294 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-8deb711a-dafb-4e5a-ac7f-f1c7d24d2e08 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:45:58.469472 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-8deb711a-dafb-4e5a-ac7f-f1c7d24d2e08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:45:58.543361 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 513/1032] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:45:58 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 10 12:45:58.544780 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaffe4a-334a-4576-b728-84bf8ec20005 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:58.568526 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deb711a-dafb-4e5a-ac7f-f1c7d24d2e08 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:58.580637 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1354aa93-70ec-43a4-8a59-0fa36a6d370a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:58.581860 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1354aa93-70ec-43a4-8a59-0fa36a6d370a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:58.582368 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1354aa93-70ec-43a4-8a59-0fa36a6d370a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:58.582756 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:58.583522 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1354aa93-70ec-43a4-8a59-0fa36a6d370a None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:58.584721 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1354aa93-70ec-43a4-8a59-0fa36a6d370a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:58.585104 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1354aa93-70ec-43a4-8a59-0fa36a6d370a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:58.586796 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1354aa93-70ec-43a4-8a59-0fa36a6d370a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:58.596095 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 520/1033] 10.222.0.27 () {56 vars in 1115 bytes} [Fri Jun 10 12:45:58 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 12:45:58.601771 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1354aa93-70ec-43a4-8a59-0fa36a6d370a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:58.643561 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:58.673587 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deb711a-dafb-4e5a-ac7f-f1c7d24d2e08 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:58.709009 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1354aa93-70ec-43a4-8a59-0fa36a6d370a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:58.710551 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d27a03-3d93-40ad-88ee-a053fdbeb08f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:58.711842 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d27a03-3d93-40ad-88ee-a053fdbeb08f None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:58.711842 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d27a03-3d93-40ad-88ee-a053fdbeb08f None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:58.712050 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d27a03-3d93-40ad-88ee-a053fdbeb08f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:58.713095 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d27a03-3d93-40ad-88ee-a053fdbeb08f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:58.714524 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d27a03-3d93-40ad-88ee-a053fdbeb08f None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:58.716368 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 514/1034] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:45:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:58.721843 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d27a03-3d93-40ad-88ee-a053fdbeb08f None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:58.745365 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e188b7e0-4ffd-4e4e-939e-96be3213bec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:58.746272 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e188b7e0-4ffd-4e4e-939e-96be3213bec0 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:58.746663 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e188b7e0-4ffd-4e4e-939e-96be3213bec0 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:58.746935 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e188b7e0-4ffd-4e4e-939e-96be3213bec0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:58.759199 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e188b7e0-4ffd-4e4e-939e-96be3213bec0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:58.760680 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e188b7e0-4ffd-4e4e-939e-96be3213bec0 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:58.778215 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-e188b7e0-4ffd-4e4e-939e-96be3213bec0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:58.778215 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:58.778215 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:58.778215 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:58.778215 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:58.778215 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:58.778215 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:58.778215 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:58.778215 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:58.778215 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:58.778215 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:58.778215 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:58.778215 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 10 12:45:58.778215 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 10 12:45:58.778215 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 10 12:45:58.779178 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 10 12:45:58.779178 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:58.779178 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:58.779178 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:58.779178 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:58.779178 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:58.779178 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:58.779178 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:58.779178 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:58.779178 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:58.779178 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:58.779178 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:58.779178 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:58.779178 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:58.779178 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:58.779178 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:58.779178 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:58.780342 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 10 12:45:58.780342 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 10 12:45:58.780342 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:58.780342 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:58.781866 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 521/1035] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:45:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 10 12:45:58.787625 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e188b7e0-4ffd-4e4e-939e-96be3213bec0 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:58.814554 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:58.839566 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d27a03-3d93-40ad-88ee-a053fdbeb08f None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:58.876595 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a177e797-8f84-49f9-9f08-6cedae9a220e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:58.877430 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a177e797-8f84-49f9-9f08-6cedae9a220e None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:58.877552 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a177e797-8f84-49f9-9f08-6cedae9a220e None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:58.877845 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a177e797-8f84-49f9-9f08-6cedae9a220e None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:58.878517 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a177e797-8f84-49f9-9f08-6cedae9a220e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:58.880609 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a177e797-8f84-49f9-9f08-6cedae9a220e None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:58.893262 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:58.903179 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e188b7e0-4ffd-4e4e-939e-96be3213bec0 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:58.937318 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb2d754-abd1-4c89-a0e0-a7b7ee36109a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:58.937904 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb2d754-abd1-4c89-a0e0-a7b7ee36109a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:58.938186 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb2d754-abd1-4c89-a0e0-a7b7ee36109a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:58.938461 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb2d754-abd1-4c89-a0e0-a7b7ee36109a None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:58.939177 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb2d754-abd1-4c89-a0e0-a7b7ee36109a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:58.939479 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb2d754-abd1-4c89-a0e0-a7b7ee36109a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:58.940863 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb2d754-abd1-4c89-a0e0-a7b7ee36109a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:58.953116 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 522/1036] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Jun 10 12:45:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 12:45:58.958285 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb2d754-abd1-4c89-a0e0-a7b7ee36109a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:58.994618 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 515/1037] 10.222.0.27 () {60 vars in 1132 bytes} [Fri Jun 10 12:45:58 2022] POST /identity/v3/users => generated 285 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 10 12:45:59.074179 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb2d754-abd1-4c89-a0e0-a7b7ee36109a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:59.091134 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7099bb-8e4d-4d1d-842b-f7856a95d132 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:59.091449 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7099bb-8e4d-4d1d-842b-f7856a95d132 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:59.091695 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7099bb-8e4d-4d1d-842b-f7856a95d132 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:59.093246 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 516/1038] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:45:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 10 12:45:59.098805 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc85944-1ba8-4f24-a1a9-4e1547ee22a2 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:59.099092 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc85944-1ba8-4f24-a1a9-4e1547ee22a2 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:59.099370 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc85944-1ba8-4f24-a1a9-4e1547ee22a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:59.111763 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d1bdbb-d0e5-4144-bab8-fa59c790df4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:59.112603 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d1bdbb-d0e5-4144-bab8-fa59c790df4c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:59.112940 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d1bdbb-d0e5-4144-bab8-fa59c790df4c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:59.113220 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d1bdbb-d0e5-4144-bab8-fa59c790df4c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:59.117254 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-fcc85944-1ba8-4f24-a1a9-4e1547ee22a2 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:45:59.118552 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-fcc85944-1ba8-4f24-a1a9-4e1547ee22a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:45:59.122226 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d1bdbb-d0e5-4144-bab8-fa59c790df4c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:59.123678 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d1bdbb-d0e5-4144-bab8-fa59c790df4c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:59.139174 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-f2d1bdbb-d0e5-4144-bab8-fa59c790df4c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 10 12:45:59.139174 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:59.139174 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:59.139174 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:59.139174 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:59.139174 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:59.139174 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:59.139174 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:59.139174 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:59.139174 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:59.139174 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:59.139174 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:59.139174 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 10 12:45:59.139174 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 10 12:45:59.139174 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 10 12:45:59.139174 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 10 12:45:59.143100 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:59.143100 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:59.143100 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 10 12:45:59.143100 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 10 12:45:59.143100 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 10 12:45:59.143100 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 10 12:45:59.143100 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:59.143100 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:59.143100 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:59.143100 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:59.143100 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:59.143100 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:59.143100 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:59.143100 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:59.143100 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:59.143100 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:59.143100 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:59.143100 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:59.144114 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:59.144114 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:59.144114 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 10 12:45:59.144114 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 10 12:45:59.144114 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 10 12:45:59.144114 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 10 12:45:59.144114 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 10 12:45:59.144114 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 10 12:45:59.144114 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 10 12:45:59.144114 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:59.144114 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 523/1039] 10.222.0.27 () {56 vars in 1093 bytes} [Fri Jun 10 12:45:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 10 12:45:59.146262 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d1bdbb-d0e5-4144-bab8-fa59c790df4c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:59.160357 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:59.194594 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 517/1040] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:45:59 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 10 12:45:59.218122 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc85944-1ba8-4f24-a1a9-4e1547ee22a2 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:59.256806 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d1bdbb-d0e5-4144-bab8-fa59c790df4c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:59.296555 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798116ea-d200-481e-93d8-45c416fce81a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:59.297184 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:59.297327 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798116ea-d200-481e-93d8-45c416fce81a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:59.297506 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798116ea-d200-481e-93d8-45c416fce81a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:59.297766 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798116ea-d200-481e-93d8-45c416fce81a None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:59.300066 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798116ea-d200-481e-93d8-45c416fce81a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:59.300251 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798116ea-d200-481e-93d8-45c416fce81a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:59.302334 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798116ea-d200-481e-93d8-45c416fce81a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:59.320657 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 524/1041] 10.222.0.27 () {56 vars in 1133 bytes} [Fri Jun 10 12:45:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 12:45:59.325983 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798116ea-d200-481e-93d8-45c416fce81a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:59.335510 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc85944-1ba8-4f24-a1a9-4e1547ee22a2 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:59.372314 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e44022b-2aa6-482f-8ef8-8cbf1f1cf544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:59.373110 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e44022b-2aa6-482f-8ef8-8cbf1f1cf544 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:59.373392 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e44022b-2aa6-482f-8ef8-8cbf1f1cf544 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:59.373678 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e44022b-2aa6-482f-8ef8-8cbf1f1cf544 None admin] PATH_INFO: `/v3/services/image` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:59.374331 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e44022b-2aa6-482f-8ef8-8cbf1f1cf544 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:59.375702 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e44022b-2aa6-482f-8ef8-8cbf1f1cf544 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:59.387302 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-2e44022b-2aa6-482f-8ef8-8cbf1f1cf544 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 10 12:45:59.387302 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:59.387302 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 10 12:45:59.387302 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 10 12:45:59.387302 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 10 12:45:59.387302 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 10 12:45:59.387302 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 10 12:45:59.387302 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 10 12:45:59.387302 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 10 12:45:59.387302 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:59.387302 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 10 12:45:59.387302 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:59.387302 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:59.387302 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:59.387302 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:59.387302 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:59.387302 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:59.388421 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:59.388421 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:59.388421 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:59.388421 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:59.388421 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:59.388421 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:59.388421 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 10 12:45:59.388421 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 10 12:45:59.388421 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 10 12:45:59.388421 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 10 12:45:59.388421 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:59.388421 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:59.388421 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:59.388421 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:59.388421 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:59.388421 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:59.389302 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:59.389302 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:59.389302 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:59.389302 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:59.389302 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:59.389302 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:59.389302 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:59.389302 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:59.389302 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:59.389302 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:59.389302 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 10 12:45:59.389302 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 10 12:45:59.389302 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 10 12:45:59.389302 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:45:59.389302 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 518/1042] 10.222.0.27 () {56 vars in 1105 bytes} [Fri Jun 10 12:45:59 2022] GET /identity/v3/services/image => generated 86 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 10 12:45:59.394131 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e44022b-2aa6-482f-8ef8-8cbf1f1cf544 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:59.399600 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:59.434345 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798116ea-d200-481e-93d8-45c416fce81a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:59.471237 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e26be3-5f37-4df2-8a3b-e88e75225282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:59.471835 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e26be3-5f37-4df2-8a3b-e88e75225282 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:59.472215 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e26be3-5f37-4df2-8a3b-e88e75225282 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:59.472510 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e26be3-5f37-4df2-8a3b-e88e75225282 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:59.508628 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e44022b-2aa6-482f-8ef8-8cbf1f1cf544 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:59.542818 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:59.548423 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e26be3-5f37-4df2-8a3b-e88e75225282 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:59.548944 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f2408c-8edf-4e1a-9cf7-9d3a28fe2b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:59.549616 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f2408c-8edf-4e1a-9cf7-9d3a28fe2b7a None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:59.549903 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f2408c-8edf-4e1a-9cf7-9d3a28fe2b7a None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:59.549996 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e26be3-5f37-4df2-8a3b-e88e75225282 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:59.550183 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f2408c-8edf-4e1a-9cf7-9d3a28fe2b7a None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:59.550893 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f2408c-8edf-4e1a-9cf7-9d3a28fe2b7a None admin] RBAC: Adding query filter params (name=image) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:59.551203 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f2408c-8edf-4e1a-9cf7-9d3a28fe2b7a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:59.553667 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f2408c-8edf-4e1a-9cf7-9d3a28fe2b7a None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:59.558120 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 519/1043] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:45:59 2022] GET /identity/v3/services?name=image => generated 122 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 12:45:59.563587 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f2408c-8edf-4e1a-9cf7-9d3a28fe2b7a None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:59.664972 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:59.702330 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 525/1044] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Jun 10 12:45:59 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 10 12:45:59.706309 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f2408c-8edf-4e1a-9cf7-9d3a28fe2b7a None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:59.708340 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e26be3-5f37-4df2-8a3b-e88e75225282 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:59.743333 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff08a1c1-e486-4a0b-abf7-594b96367386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:59.743965 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff08a1c1-e486-4a0b-abf7-594b96367386 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:59.744254 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff08a1c1-e486-4a0b-abf7-594b96367386 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:59.744537 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff08a1c1-e486-4a0b-abf7-594b96367386 None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:59.745256 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff08a1c1-e486-4a0b-abf7-594b96367386 None admin] RBAC: Adding query filter params (type=image) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:45:59.745569 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff08a1c1-e486-4a0b-abf7-594b96367386 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:59.746962 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff08a1c1-e486-4a0b-abf7-594b96367386 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:59.752077 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 520/1045] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:45:59 2022] GET /identity/v3/services?type=image => generated 350 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:45:59.757792 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff08a1c1-e486-4a0b-abf7-594b96367386 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:59.822498 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e26be3-5f37-4df2-8a3b-e88e75225282 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:59.824124 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:45:59.863568 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4e528c-fe15-4883-a926-278e050258e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:59.864439 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4e528c-fe15-4883-a926-278e050258e7 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:59.864731 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4e528c-fe15-4883-a926-278e050258e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:59.865015 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4e528c-fe15-4883-a926-278e050258e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:59.876216 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4e528c-fe15-4883-a926-278e050258e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:59.876819 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff08a1c1-e486-4a0b-abf7-594b96367386 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:45:59.877658 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4e528c-fe15-4883-a926-278e050258e7 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:59.898177 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-cf4e528c-fe15-4883-a926-278e050258e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:59.898177 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:45:59.898177 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:45:59.898177 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:45:59.898177 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:45:59.898177 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:45:59.898177 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:45:59.898177 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:45:59.898177 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:45:59.898177 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:45:59.898177 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:45:59.898177 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:45:59.898177 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 10 12:45:59.898177 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 10 12:45:59.898177 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 10 12:45:59.899217 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 10 12:45:59.899217 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:45:59.899217 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:45:59.899217 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:45:59.899217 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:45:59.899217 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:45:59.899217 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:45:59.899217 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:45:59.899217 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:45:59.899217 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:45:59.899217 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:45:59.899217 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:45:59.899217 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:45:59.899217 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:45:59.899217 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:45:59.899217 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:45:59.899217 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:45:59.900191 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 10 12:45:59.900191 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 10 12:45:59.900191 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:45:59.900191 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:45:59.900191 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 526/1046] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:45:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 10 12:45:59.905071 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4e528c-fe15-4883-a926-278e050258e7 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:45:59.915753 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4e3ac3-758b-4f4a-b1bf-bf27a860b3f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:45:59.916454 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4e3ac3-758b-4f4a-b1bf-bf27a860b3f7 None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:45:59.916746 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4e3ac3-758b-4f4a-b1bf-bf27a860b3f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:45:59.917023 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4e3ac3-758b-4f4a-b1bf-bf27a860b3f7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:45:59.917759 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4e3ac3-758b-4f4a-b1bf-bf27a860b3f7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:45:59.919146 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4e3ac3-758b-4f4a-b1bf-bf27a860b3f7 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:45:59.928695 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:46:00.011979 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 521/1047] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Jun 10 12:45:59 2022] POST /identity/v3/endpoints => generated 327 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 10 12:46:00.012188 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4e528c-fe15-4883-a926-278e050258e7 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:00.050430 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38418212-31d9-44a0-a9b3-8f66459101db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:00.051119 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38418212-31d9-44a0-a9b3-8f66459101db None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:00.051403 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38418212-31d9-44a0-a9b3-8f66459101db None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:00.051678 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38418212-31d9-44a0-a9b3-8f66459101db None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:00.052448 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38418212-31d9-44a0-a9b3-8f66459101db None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:46:00.052758 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38418212-31d9-44a0-a9b3-8f66459101db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:00.054184 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38418212-31d9-44a0-a9b3-8f66459101db None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:00.067076 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 527/1048] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Jun 10 12:45:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 12:46:00.073130 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4e3ac3-758b-4f4a-b1bf-bf27a860b3f7 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:00.188683 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4e3ac3-758b-4f4a-b1bf-bf27a860b3f7 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:00.205423 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:46:00.228847 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c66b8ac-1aaf-4092-a428-bcdd0b45cdad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:00.229655 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c66b8ac-1aaf-4092-a428-bcdd0b45cdad None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:00.229943 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c66b8ac-1aaf-4092-a428-bcdd0b45cdad None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:00.230254 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c66b8ac-1aaf-4092-a428-bcdd0b45cdad None admin] PATH_INFO: `/v3/projects/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:00.237873 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c66b8ac-1aaf-4092-a428-bcdd0b45cdad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:00.241962 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c66b8ac-1aaf-4092-a428-bcdd0b45cdad None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:00.254042 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-6c66b8ac-1aaf-4092-a428-bcdd0b45cdad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 10 12:46:00.254042 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:46:00.254042 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:46:00.254042 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:46:00.254042 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:46:00.254042 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:46:00.254042 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:46:00.254042 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:46:00.254042 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:46:00.254042 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:46:00.254042 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:46:00.254042 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:46:00.254042 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 10 12:46:00.254042 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 10 12:46:00.254042 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 10 12:46:00.255535 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 10 12:46:00.255535 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:46:00.255535 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:46:00.255535 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:46:00.255535 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:46:00.255535 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:46:00.255535 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:46:00.255535 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:46:00.255535 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:46:00.255535 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:46:00.255535 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:46:00.255535 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:46:00.255535 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:46:00.255535 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:46:00.255535 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:46:00.255535 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:46:00.255535 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:46:00.256498 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 10 12:46:00.256498 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 10 12:46:00.256498 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 10 12:46:00.256498 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 10 12:46:00.256498 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 10 12:46:00.256498 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 10 12:46:00.256498 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 10 12:46:00.256498 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:46:00.256930 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 522/1049] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Jun 10 12:46:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 10 12:46:00.262348 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38418212-31d9-44a0-a9b3-8f66459101db None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:00.340980 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:46:00.363888 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38418212-31d9-44a0-a9b3-8f66459101db None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:00.399386 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c14012-8a7c-480f-bd96-1b7253f1ddd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:00.400081 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c14012-8a7c-480f-bd96-1b7253f1ddd1 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:00.400373 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c14012-8a7c-480f-bd96-1b7253f1ddd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:00.400650 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c14012-8a7c-480f-bd96-1b7253f1ddd1 None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:00.401440 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c14012-8a7c-480f-bd96-1b7253f1ddd1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:46:00.401752 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c14012-8a7c-480f-bd96-1b7253f1ddd1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:00.403185 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c14012-8a7c-480f-bd96-1b7253f1ddd1 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:00.420685 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 528/1050] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Jun 10 12:46:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:46:00.421878 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c66b8ac-1aaf-4092-a428-bcdd0b45cdad None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:00.524724 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c66b8ac-1aaf-4092-a428-bcdd0b45cdad None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:00.559661 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e28e29-2fc8-4338-b145-53e41938d082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:00.560344 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e28e29-2fc8-4338-b145-53e41938d082 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:00.560632 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e28e29-2fc8-4338-b145-53e41938d082 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:00.560910 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e28e29-2fc8-4338-b145-53e41938d082 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:00.561750 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e28e29-2fc8-4338-b145-53e41938d082 None admin] RBAC: Adding query filter params (role.id=55ae4712a75e4f9bb542575264d55f64, scope.project.id=73ea5b23a2d34544ab493f4c5e4034b8, user.id=8bd6caa461e84100a3d5fe5af5d80f3e) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:46:00.562064 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e28e29-2fc8-4338-b145-53e41938d082 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:00.563690 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e28e29-2fc8-4338-b145-53e41938d082 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:00.575949 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 523/1051] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Jun 10 12:46:00 2022] GET /identity/v3/role_assignments?user.id=8bd6caa461e84100a3d5fe5af5d80f3e&scope.project.id=73ea5b23a2d34544ab493f4c5e4034b8&role.id=55ae4712a75e4f9bb542575264d55f64 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:46:01.239662 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baab1da-11ee-4909-8702-e85591bbcaf9 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:01.240046 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baab1da-11ee-4909-8702-e85591bbcaf9 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:01.240227 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baab1da-11ee-4909-8702-e85591bbcaf9 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:01.241742 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 529/1052] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:46:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:46:01.246720 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a48e10-18ed-4833-9ca4-8fdb7daaf9d0 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:01.247154 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a48e10-18ed-4833-9ca4-8fdb7daaf9d0 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:01.247335 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a48e10-18ed-4833-9ca4-8fdb7daaf9d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:01.264370 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-50a48e10-18ed-4833-9ca4-8fdb7daaf9d0 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:46:01.265566 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-50a48e10-18ed-4833-9ca4-8fdb7daaf9d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:46:01.296620 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:46:01.341246 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 524/1053] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:46:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 10 12:46:01.374628 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baab1da-11ee-4909-8702-e85591bbcaf9 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:01.476433 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baab1da-11ee-4909-8702-e85591bbcaf9 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:01.509565 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fbc516-d5a9-4ec1-8b24-e3b14cb3aeaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:01.510296 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fbc516-d5a9-4ec1-8b24-e3b14cb3aeaa None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:01.510593 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fbc516-d5a9-4ec1-8b24-e3b14cb3aeaa None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:01.510865 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fbc516-d5a9-4ec1-8b24-e3b14cb3aeaa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:01.513502 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:46:01.521889 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fbc516-d5a9-4ec1-8b24-e3b14cb3aeaa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:46:01.529768 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-b0fbc516-d5a9-4ec1-8b24-e3b14cb3aeaa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 10 12:46:01.529768 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:46:01.529768 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:46:01.529768 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:46:01.529768 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:46:01.529768 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:46:01.529768 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:46:01.529768 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:46:01.529768 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:46:01.529768 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:46:01.529768 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:46:01.529768 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:46:01.529768 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:46:01.529768 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:46:01.529768 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:46:01.529768 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:46:01.530854 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:46:01.530854 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:46:01.530854 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:46:01.530854 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:46:01.530854 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:46:01.530854 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:46:01.530854 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:46:01.530854 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:46:01.530854 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:46:01.530854 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:46:01.530854 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:46:01.530854 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:46:01.530854 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:46:01.530854 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:46:01.530854 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:46:01.530854 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:46:01.531753 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:46:01.531753 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:46:01.531753 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:46:01.531753 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:46:01.531753 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:46:01.531753 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:46:01.531753 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:46:01.531753 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:46:01.531753 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:46:01.531753 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:46:01.531753 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 10 12:46:01.531753 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:46:01.535466 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 530/1054] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Jun 10 12:46:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:46:01.541236 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a48e10-18ed-4833-9ca4-8fdb7daaf9d0 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:01.644048 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a48e10-18ed-4833-9ca4-8fdb7daaf9d0 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:01.680271 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4736f3-2b2e-4821-9aae-d4850a6e45cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:01.680957 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4736f3-2b2e-4821-9aae-d4850a6e45cf None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:01.681234 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4736f3-2b2e-4821-9aae-d4850a6e45cf None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:01.681494 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4736f3-2b2e-4821-9aae-d4850a6e45cf None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:01.682469 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4736f3-2b2e-4821-9aae-d4850a6e45cf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:46:01.682771 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4736f3-2b2e-4821-9aae-d4850a6e45cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:01.684163 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4736f3-2b2e-4821-9aae-d4850a6e45cf None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:01.698229 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 525/1055] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Jun 10 12:46:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:46:01.699675 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fbc516-d5a9-4ec1-8b24-e3b14cb3aeaa None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:01.804191 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fbc516-d5a9-4ec1-8b24-e3b14cb3aeaa None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:01.837980 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743855a2-ad73-4efa-bc3b-3cada29a01a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:01.838848 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743855a2-ad73-4efa-bc3b-3cada29a01a9 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:01.839185 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743855a2-ad73-4efa-bc3b-3cada29a01a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:01.839466 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743855a2-ad73-4efa-bc3b-3cada29a01a9 None admin] PATH_INFO: `/v3/users/099f629ee67840b9901c7ef6d7bfe89e` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:01.858575 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743855a2-ad73-4efa-bc3b-3cada29a01a9 None admin] RBAC: Authorizing `identity:get_user(user_id=099f629ee67840b9901c7ef6d7bfe89e)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:01.860026 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743855a2-ad73-4efa-bc3b-3cada29a01a9 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:01.865158 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 531/1056] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Jun 10 12:46:01 2022] GET /identity/v3/users/099f629ee67840b9901c7ef6d7bfe89e => generated 285 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:46:01.868524 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4736f3-2b2e-4821-9aae-d4850a6e45cf None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:01.901370 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:46:01.971061 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4736f3-2b2e-4821-9aae-d4850a6e45cf None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:02.005390 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80009d62-bc23-4389-8af9-39907bde491c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:02.006043 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80009d62-bc23-4389-8af9-39907bde491c None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:02.006329 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80009d62-bc23-4389-8af9-39907bde491c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:02.006632 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80009d62-bc23-4389-8af9-39907bde491c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:02.079048 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80009d62-bc23-4389-8af9-39907bde491c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:02.080466 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80009d62-bc23-4389-8af9-39907bde491c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:02.110522 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:46:02.173370 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c115bb4-b203-48d0-934e-80d3fa74a80e None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:02.173714 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c115bb4-b203-48d0-934e-80d3fa74a80e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:02.173936 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c115bb4-b203-48d0-934e-80d3fa74a80e None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:02.175295 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 532/1057] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:46:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 10 12:46:02.180777 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c8f027-6056-4a06-a068-65e6c76a2203 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:02.181062 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c8f027-6056-4a06-a068-65e6c76a2203 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:02.181376 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c8f027-6056-4a06-a068-65e6c76a2203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:02.189812 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 526/1058] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Jun 10 12:46:01 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 10 12:46:02.196553 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80009d62-bc23-4389-8af9-39907bde491c None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:02.198255 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-50c8f027-6056-4a06-a068-65e6c76a2203 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:46:02.199443 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-50c8f027-6056-4a06-a068-65e6c76a2203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:46:02.252221 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:46:02.271329 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 533/1059] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:46:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 10 12:46:02.294120 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c8f027-6056-4a06-a068-65e6c76a2203 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:02.307465 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80009d62-bc23-4389-8af9-39907bde491c None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:02.340957 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:46:02.346687 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82ad260-7025-4174-9c42-ae61bffbc0d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:02.347559 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82ad260-7025-4174-9c42-ae61bffbc0d2 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:02.351618 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82ad260-7025-4174-9c42-ae61bffbc0d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:02.351618 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82ad260-7025-4174-9c42-ae61bffbc0d2 None admin] PATH_INFO: `/v3/projects/e9d856a2f36941c9a3e6819085041374` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:02.351618 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82ad260-7025-4174-9c42-ae61bffbc0d2 None admin] RBAC: Authorizing `identity:get_project(project_id=e9d856a2f36941c9a3e6819085041374)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:02.351618 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82ad260-7025-4174-9c42-ae61bffbc0d2 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:02.353108 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 527/1060] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Jun 10 12:46:02 2022] GET /identity/v3/projects/e9d856a2f36941c9a3e6819085041374 => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 12:46:02.359027 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82ad260-7025-4174-9c42-ae61bffbc0d2 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:02.401191 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c8f027-6056-4a06-a068-65e6c76a2203 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:02.436718 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0011a01d-3d79-4f6c-bb19-b58b56021fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:02.437371 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0011a01d-3d79-4f6c-bb19-b58b56021fec None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:02.437658 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0011a01d-3d79-4f6c-bb19-b58b56021fec None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:02.437939 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0011a01d-3d79-4f6c-bb19-b58b56021fec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:02.468067 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82ad260-7025-4174-9c42-ae61bffbc0d2 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:02.493720 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:46:02.506763 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb465c1-0b4d-40e8-ad0a-911b6d347a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:02.508106 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb465c1-0b4d-40e8-ad0a-911b6d347a66 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:02.508800 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb465c1-0b4d-40e8-ad0a-911b6d347a66 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:02.509253 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb465c1-0b4d-40e8-ad0a-911b6d347a66 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:02.509370 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0011a01d-3d79-4f6c-bb19-b58b56021fec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:02.510322 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb465c1-0b4d-40e8-ad0a-911b6d347a66 None admin] RBAC: Adding query filter params (role.id=b43d5a8247e244869c2c70f986f42e8a, scope.project.id=e9d856a2f36941c9a3e6819085041374, user.id=099f629ee67840b9901c7ef6d7bfe89e) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:46:02.510717 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0011a01d-3d79-4f6c-bb19-b58b56021fec None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:02.510927 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb465c1-0b4d-40e8-ad0a-911b6d347a66 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:02.512671 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb465c1-0b4d-40e8-ad0a-911b6d347a66 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:02.519812 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 528/1061] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Jun 10 12:46:02 2022] GET /identity/v3/role_assignments?user.id=099f629ee67840b9901c7ef6d7bfe89e&scope.project.id=e9d856a2f36941c9a3e6819085041374&role.id=b43d5a8247e244869c2c70f986f42e8a => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:46:02.623426 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 534/1062] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Jun 10 12:46:02 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 10 12:46:02.629670 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb465c1-0b4d-40e8-ad0a-911b6d347a66 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:02.679316 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ab4573-98fd-48e6-912e-71773cf341e3 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:02.680073 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ab4573-98fd-48e6-912e-71773cf341e3 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:02.680460 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ab4573-98fd-48e6-912e-71773cf341e3 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:02.681875 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 535/1063] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:46:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 10 12:46:02.687537 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174332e1-0105-48d2-a013-ffa5c373548f None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:02.687824 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174332e1-0105-48d2-a013-ffa5c373548f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:02.688139 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174332e1-0105-48d2-a013-ffa5c373548f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:02.697858 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:46:02.705284 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-174332e1-0105-48d2-a013-ffa5c373548f None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:46:02.706488 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-174332e1-0105-48d2-a013-ffa5c373548f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:46:02.739771 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb465c1-0b4d-40e8-ad0a-911b6d347a66 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:02.774476 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b165993a-5867-47aa-aaed-78b648c22ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:02.775249 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b165993a-5867-47aa-aaed-78b648c22ba6 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:02.775531 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b165993a-5867-47aa-aaed-78b648c22ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:02.775803 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b165993a-5867-47aa-aaed-78b648c22ba6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:02.779428 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 536/1064] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:46:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 10 12:46:02.789238 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b165993a-5867-47aa-aaed-78b648c22ba6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:02.790687 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b165993a-5867-47aa-aaed-78b648c22ba6 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:02.802629 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174332e1-0105-48d2-a013-ffa5c373548f None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:02.807794 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-b165993a-5867-47aa-aaed-78b648c22ba6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:46:02.807794 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:46:02.807794 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:46:02.807794 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:46:02.807794 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:46:02.807794 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:46:02.807794 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:46:02.807794 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:46:02.807794 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:46:02.807794 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:46:02.807794 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:46:02.807794 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:46:02.807794 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 10 12:46:02.807794 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 10 12:46:02.807794 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 10 12:46:02.808863 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 10 12:46:02.808863 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:46:02.808863 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:46:02.808863 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:46:02.808863 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:46:02.808863 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:46:02.808863 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:46:02.808863 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:46:02.808863 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:46:02.808863 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:46:02.808863 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:46:02.808863 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:46:02.808863 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:46:02.808863 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:46:02.808863 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:46:02.808863 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:46:02.808863 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:46:02.809774 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 10 12:46:02.809774 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 10 12:46:02.809774 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:46:02.809774 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:46:02.809774 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 529/1065] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:46:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:46:02.817274 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b165993a-5867-47aa-aaed-78b648c22ba6 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:02.822114 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:46:02.919947 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174332e1-0105-48d2-a013-ffa5c373548f None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:02.930831 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b165993a-5867-47aa-aaed-78b648c22ba6 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:02.944797 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:46:02.957033 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe7548b-aaff-4209-8b20-d7d5a7e0ad9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:02.958632 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe7548b-aaff-4209-8b20-d7d5a7e0ad9c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:02.958914 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe7548b-aaff-4209-8b20-d7d5a7e0ad9c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:02.958998 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe7548b-aaff-4209-8b20-d7d5a7e0ad9c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:02.966047 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99bc26f-9efc-462e-a8c1-5d3389b02bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:02.966709 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99bc26f-9efc-462e-a8c1-5d3389b02bcb None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:02.966992 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99bc26f-9efc-462e-a8c1-5d3389b02bcb None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:02.967270 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99bc26f-9efc-462e-a8c1-5d3389b02bcb None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:02.968035 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99bc26f-9efc-462e-a8c1-5d3389b02bcb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:46:02.968341 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99bc26f-9efc-462e-a8c1-5d3389b02bcb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:02.969724 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99bc26f-9efc-462e-a8c1-5d3389b02bcb None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:02.970681 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe7548b-aaff-4209-8b20-d7d5a7e0ad9c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:46:02.979539 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-bbe7548b-aaff-4209-8b20-d7d5a7e0ad9c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 10 12:46:02.979539 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:46:02.979539 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:46:02.979539 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:46:02.979539 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:46:02.979539 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:46:02.979539 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:46:02.979539 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:46:02.979539 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:46:02.979539 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:46:02.979539 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:46:02.979539 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:46:02.979539 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:46:02.979539 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:46:02.979539 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:46:02.979539 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:46:02.980673 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:46:02.980673 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:46:02.980673 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:46:02.980673 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:46:02.980673 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:46:02.980673 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:46:02.980673 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:46:02.980673 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:46:02.980673 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:46:02.980673 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:46:02.980673 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:46:02.980673 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:46:02.980673 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:46:02.980673 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:46:02.980673 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:46:02.980673 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:46:02.981727 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:46:02.981727 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:46:02.981727 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:46:02.981727 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:46:02.981727 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:46:02.981727 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:46:02.981727 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:46:02.981727 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:46:02.981727 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:46:02.981727 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:46:02.981727 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 10 12:46:02.981727 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:46:02.983325 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 530/1066] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Jun 10 12:46:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 12:46:02.984334 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 537/1067] 10.222.0.27 () {56 vars in 1120 bytes} [Fri Jun 10 12:46:02 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 10 12:46:02.990155 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99bc26f-9efc-462e-a8c1-5d3389b02bcb None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:03.100252 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99bc26f-9efc-462e-a8c1-5d3389b02bcb None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:03.136425 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb1481d-53ab-4220-82e6-4d751acdd8f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:03.137107 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb1481d-53ab-4220-82e6-4d751acdd8f9 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:03.137389 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb1481d-53ab-4220-82e6-4d751acdd8f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:03.137662 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb1481d-53ab-4220-82e6-4d751acdd8f9 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:03.138418 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb1481d-53ab-4220-82e6-4d751acdd8f9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:46:03.138718 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb1481d-53ab-4220-82e6-4d751acdd8f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:03.140109 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb1481d-53ab-4220-82e6-4d751acdd8f9 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:03.150113 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 531/1068] 10.222.0.27 () {56 vars in 1115 bytes} [Fri Jun 10 12:46:02 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:46:03.155500 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe7548b-aaff-4209-8b20-d7d5a7e0ad9c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:03.235209 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:46:03.262804 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe7548b-aaff-4209-8b20-d7d5a7e0ad9c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:03.299540 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c8fa63-5e42-4d4c-b302-6f56f0dc70aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:03.300536 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c8fa63-5e42-4d4c-b302-6f56f0dc70aa None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:03.300816 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c8fa63-5e42-4d4c-b302-6f56f0dc70aa None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:03.301084 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c8fa63-5e42-4d4c-b302-6f56f0dc70aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:03.311611 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c8fa63-5e42-4d4c-b302-6f56f0dc70aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:03.313089 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c8fa63-5e42-4d4c-b302-6f56f0dc70aa None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:03.330022 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-a1c8fa63-5e42-4d4c-b302-6f56f0dc70aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:46:03.330022 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:46:03.330022 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:46:03.330022 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:46:03.330022 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:46:03.330022 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:46:03.330022 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:46:03.330022 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:46:03.330022 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:46:03.330022 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:46:03.330022 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:46:03.330022 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:46:03.330022 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 10 12:46:03.330022 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 10 12:46:03.330022 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 10 12:46:03.330950 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 10 12:46:03.330950 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:46:03.330950 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:46:03.330950 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:46:03.330950 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:46:03.330950 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:46:03.330950 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:46:03.330950 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:46:03.330950 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:46:03.330950 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:46:03.330950 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:46:03.330950 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:46:03.330950 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:46:03.330950 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:46:03.330950 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:46:03.330950 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:46:03.330950 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:46:03.331906 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 10 12:46:03.331906 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 10 12:46:03.331906 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:46:03.331906 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:46:03.331906 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 538/1069] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:46:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:46:03.337438 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb1481d-53ab-4220-82e6-4d751acdd8f9 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:03.372221 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:46:03.434798 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb1481d-53ab-4220-82e6-4d751acdd8f9 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:03.467588 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9c76a0-d047-4bca-a314-5ba19483d972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:03.468263 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9c76a0-d047-4bca-a314-5ba19483d972 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:03.468545 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9c76a0-d047-4bca-a314-5ba19483d972 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:03.468816 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9c76a0-d047-4bca-a314-5ba19483d972 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:03.469523 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9c76a0-d047-4bca-a314-5ba19483d972 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:46:03.469826 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9c76a0-d047-4bca-a314-5ba19483d972 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:03.471198 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9c76a0-d047-4bca-a314-5ba19483d972 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:03.486426 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 532/1070] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Jun 10 12:46:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:46:03.487462 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c8fa63-5e42-4d4c-b302-6f56f0dc70aa None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:03.584594 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c8fa63-5e42-4d4c-b302-6f56f0dc70aa None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:03.609112 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:46:03.618730 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79de2d68-aec9-48dc-9e1d-3a012ea41642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:03.619542 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79de2d68-aec9-48dc-9e1d-3a012ea41642 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:03.619824 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79de2d68-aec9-48dc-9e1d-3a012ea41642 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:03.620129 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79de2d68-aec9-48dc-9e1d-3a012ea41642 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:03.628169 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79de2d68-aec9-48dc-9e1d-3a012ea41642 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:03.630167 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79de2d68-aec9-48dc-9e1d-3a012ea41642 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:03.644241 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-79de2d68-aec9-48dc-9e1d-3a012ea41642 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 10 12:46:03.644241 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:46:03.644241 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:46:03.644241 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:46:03.644241 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:46:03.644241 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:46:03.644241 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:46:03.644241 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:46:03.644241 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:46:03.644241 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:46:03.644241 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:46:03.644241 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:46:03.644241 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 10 12:46:03.644241 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 10 12:46:03.644241 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 10 12:46:03.644241 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 10 12:46:03.645430 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:46:03.645430 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:46:03.645430 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 10 12:46:03.645430 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 10 12:46:03.645430 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 10 12:46:03.645430 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 10 12:46:03.645430 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:46:03.645430 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:46:03.645430 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:46:03.645430 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:46:03.645430 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:46:03.645430 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:46:03.645430 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:46:03.645430 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:46:03.645430 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:46:03.645430 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:46:03.645430 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:46:03.645430 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:46:03.646775 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:46:03.646775 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:46:03.646775 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 10 12:46:03.646775 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 10 12:46:03.646775 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 10 12:46:03.646775 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 10 12:46:03.646775 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 10 12:46:03.646775 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 10 12:46:03.646775 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 10 12:46:03.646775 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:46:03.646775 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 539/1071] 10.222.0.27 () {56 vars in 1093 bytes} [Fri Jun 10 12:46:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 10 12:46:03.651116 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9c76a0-d047-4bca-a314-5ba19483d972 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:03.742178 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:46:03.747740 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9c76a0-d047-4bca-a314-5ba19483d972 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:03.780331 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c69552-f6f7-4cb0-a23b-24002ebebe8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:03.780997 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c69552-f6f7-4cb0-a23b-24002ebebe8b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:03.781277 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c69552-f6f7-4cb0-a23b-24002ebebe8b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:03.781550 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c69552-f6f7-4cb0-a23b-24002ebebe8b None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:03.782374 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c69552-f6f7-4cb0-a23b-24002ebebe8b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:46:03.782680 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c69552-f6f7-4cb0-a23b-24002ebebe8b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:03.784128 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c69552-f6f7-4cb0-a23b-24002ebebe8b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:03.801250 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 533/1072] 10.222.0.27 () {56 vars in 1133 bytes} [Fri Jun 10 12:46:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:46:03.807309 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79de2d68-aec9-48dc-9e1d-3a012ea41642 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:03.902366 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79de2d68-aec9-48dc-9e1d-3a012ea41642 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:03.935411 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ff594b-2b92-4d5a-8784-a1a173634817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:03.936217 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ff594b-2b92-4d5a-8784-a1a173634817 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:03.936499 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ff594b-2b92-4d5a-8784-a1a173634817 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:03.936772 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ff594b-2b92-4d5a-8784-a1a173634817 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:03.946922 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ff594b-2b92-4d5a-8784-a1a173634817 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:03.948376 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ff594b-2b92-4d5a-8784-a1a173634817 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:03.964741 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-59ff594b-2b92-4d5a-8784-a1a173634817 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:46:03.964741 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:46:03.964741 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:46:03.964741 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:46:03.964741 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:46:03.964741 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:46:03.964741 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:46:03.964741 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:46:03.964741 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:46:03.964741 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:46:03.964741 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:46:03.964741 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:46:03.964741 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 10 12:46:03.964741 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 10 12:46:03.964741 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 10 12:46:03.965618 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 10 12:46:03.965618 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:46:03.965618 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:46:03.965618 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:46:03.965618 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:46:03.965618 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:46:03.965618 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:46:03.965618 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:46:03.965618 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:46:03.965618 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:46:03.965618 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:46:03.965618 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:46:03.965618 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:46:03.965618 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:46:03.965618 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:46:03.965618 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:46:03.965618 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:46:03.966773 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 10 12:46:03.966773 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 10 12:46:03.966773 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:46:03.966773 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:46:03.966773 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 540/1073] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:46:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 10 12:46:03.971473 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c69552-f6f7-4cb0-a23b-24002ebebe8b None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:04.068553 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c69552-f6f7-4cb0-a23b-24002ebebe8b None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:04.101360 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671208ca-8e99-443d-b484-93878e66cac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:04.102013 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671208ca-8e99-443d-b484-93878e66cac6 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:04.102295 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671208ca-8e99-443d-b484-93878e66cac6 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:04.102565 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671208ca-8e99-443d-b484-93878e66cac6 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:04.103274 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671208ca-8e99-443d-b484-93878e66cac6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:46:04.103574 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671208ca-8e99-443d-b484-93878e66cac6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:04.104972 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671208ca-8e99-443d-b484-93878e66cac6 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:04.116124 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 534/1074] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Jun 10 12:46:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:46:04.121575 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ff594b-2b92-4d5a-8784-a1a173634817 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:04.161448 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:46:04.219342 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ff594b-2b92-4d5a-8784-a1a173634817 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:04.251709 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9e4d1b-57a3-4f1d-9d79-aab3d060c949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:04.252532 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9e4d1b-57a3-4f1d-9d79-aab3d060c949 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:04.252816 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9e4d1b-57a3-4f1d-9d79-aab3d060c949 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:04.253088 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9e4d1b-57a3-4f1d-9d79-aab3d060c949 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:04.260532 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9e4d1b-57a3-4f1d-9d79-aab3d060c949 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:04.261948 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9e4d1b-57a3-4f1d-9d79-aab3d060c949 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:04.276151 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-fc9e4d1b-57a3-4f1d-9d79-aab3d060c949 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 10 12:46:04.276151 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:46:04.276151 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:46:04.276151 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:46:04.276151 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:46:04.276151 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:46:04.276151 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:46:04.276151 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:46:04.276151 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:46:04.276151 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:46:04.276151 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:46:04.276151 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:46:04.276151 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 10 12:46:04.276151 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 10 12:46:04.276151 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 10 12:46:04.277220 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 10 12:46:04.277220 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:46:04.277220 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:46:04.277220 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:46:04.277220 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:46:04.277220 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:46:04.277220 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:46:04.277220 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:46:04.277220 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:46:04.277220 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:46:04.277220 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:46:04.277220 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:46:04.277220 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:46:04.277220 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:46:04.277220 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:46:04.277220 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:46:04.277220 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:46:04.278150 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 10 12:46:04.278150 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 10 12:46:04.278150 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 10 12:46:04.278150 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 10 12:46:04.278150 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 10 12:46:04.278150 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 10 12:46:04.278150 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 10 12:46:04.278150 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:46:04.278150 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 541/1075] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Jun 10 12:46:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:46:04.282990 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671208ca-8e99-443d-b484-93878e66cac6 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:04.299619 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:46:04.379533 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671208ca-8e99-443d-b484-93878e66cac6 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:04.412232 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea622319-88c5-4099-87d5-1da75fcb443c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:04.412877 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea622319-88c5-4099-87d5-1da75fcb443c None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:04.413154 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea622319-88c5-4099-87d5-1da75fcb443c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:04.413424 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea622319-88c5-4099-87d5-1da75fcb443c None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:04.414197 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea622319-88c5-4099-87d5-1da75fcb443c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:46:04.414499 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea622319-88c5-4099-87d5-1da75fcb443c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:04.415938 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea622319-88c5-4099-87d5-1da75fcb443c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:04.427558 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 535/1076] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Jun 10 12:46:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 12:46:04.433421 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9e4d1b-57a3-4f1d-9d79-aab3d060c949 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:04.528703 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:46:04.532768 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9e4d1b-57a3-4f1d-9d79-aab3d060c949 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:04.566104 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac69cb1-ab20-451d-948d-ea21c1593282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:04.567558 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac69cb1-ab20-451d-948d-ea21c1593282 None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:04.567841 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac69cb1-ab20-451d-948d-ea21c1593282 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:04.568144 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac69cb1-ab20-451d-948d-ea21c1593282 None admin] PATH_INFO: `/v3/projects/73ea5b23a2d34544ab493f4c5e4034b8/users/8bd6caa461e84100a3d5fe5af5d80f3e/roles/55ae4712a75e4f9bb542575264d55f64` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:04.570434 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac69cb1-ab20-451d-948d-ea21c1593282 None admin] RBAC: Authorizing `identity:create_grant(project_id=73ea5b23a2d34544ab493f4c5e4034b8, user_id=8bd6caa461e84100a3d5fe5af5d80f3e, role_id=55ae4712a75e4f9bb542575264d55f64)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:04.571938 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac69cb1-ab20-451d-948d-ea21c1593282 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:04.640210 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f6d3e0-b607-4de8-9527-cd8eb3ec2038 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:04.640439 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f6d3e0-b607-4de8-9527-cd8eb3ec2038 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:04.640684 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f6d3e0-b607-4de8-9527-cd8eb3ec2038 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:04.642047 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 536/1077] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:46:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 10 12:46:04.647370 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d62653f-d3f1-405a-afb0-790d917bdd3e None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:04.647654 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d62653f-d3f1-405a-afb0-790d917bdd3e None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:04.647962 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d62653f-d3f1-405a-afb0-790d917bdd3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:04.652142 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 542/1078] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Jun 10 12:46:04 2022] PUT /identity/v3/projects/73ea5b23a2d34544ab493f4c5e4034b8/users/8bd6caa461e84100a3d5fe5af5d80f3e/roles/55ae4712a75e4f9bb542575264d55f64 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 10 12:46:04.665232 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-8d62653f-d3f1-405a-afb0-790d917bdd3e None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:46:04.666447 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-8d62653f-d3f1-405a-afb0-790d917bdd3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:46:04.739372 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 537/1079] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:46:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 10 12:46:04.762434 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac69cb1-ab20-451d-948d-ea21c1593282 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:04.864897 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac69cb1-ab20-451d-948d-ea21c1593282 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:04.900043 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbc3021-15c0-412f-bbe3-a033d8579580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:04.900765 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbc3021-15c0-412f-bbe3-a033d8579580 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:04.901055 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbc3021-15c0-412f-bbe3-a033d8579580 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:04.901336 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbc3021-15c0-412f-bbe3-a033d8579580 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:04.912493 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbc3021-15c0-412f-bbe3-a033d8579580 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:46:04.920174 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-ecbc3021-15c0-412f-bbe3-a033d8579580 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 10 12:46:04.920174 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:46:04.920174 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:46:04.920174 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:46:04.920174 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:46:04.920174 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:46:04.920174 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:46:04.920174 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:46:04.920174 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:46:04.920174 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:46:04.920174 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:46:04.920174 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:46:04.920174 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:46:04.920174 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:46:04.920174 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:46:04.920174 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:46:04.921304 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:46:04.921304 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:46:04.921304 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:46:04.921304 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:46:04.921304 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:46:04.921304 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:46:04.921304 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:46:04.921304 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:46:04.921304 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:46:04.921304 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:46:04.921304 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:46:04.921304 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:46:04.921304 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:46:04.921304 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:46:04.921304 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:46:04.921304 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:46:04.927569 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:46:04.927569 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:46:04.927569 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:46:04.927569 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:46:04.927569 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:46:04.927569 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:46:04.927569 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:46:04.927569 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:46:04.927569 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:46:04.927569 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:46:04.927569 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 10 12:46:04.927569 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:46:04.927569 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 543/1080] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Jun 10 12:46:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:46:04.928933 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d62653f-d3f1-405a-afb0-790d917bdd3e None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:04.930431 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:46:05.024540 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d62653f-d3f1-405a-afb0-790d917bdd3e None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:05.057948 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4488b9-b12a-48d3-b9ab-98baa3a9969c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:05.058613 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4488b9-b12a-48d3-b9ab-98baa3a9969c None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:05.058895 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4488b9-b12a-48d3-b9ab-98baa3a9969c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:05.059173 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4488b9-b12a-48d3-b9ab-98baa3a9969c None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:05.059942 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4488b9-b12a-48d3-b9ab-98baa3a9969c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:46:05.060249 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4488b9-b12a-48d3-b9ab-98baa3a9969c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:05.061604 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4488b9-b12a-48d3-b9ab-98baa3a9969c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:05.069611 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 538/1081] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Jun 10 12:46:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:46:05.075328 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbc3021-15c0-412f-bbe3-a033d8579580 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:05.095253 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:46:05.174714 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbc3021-15c0-412f-bbe3-a033d8579580 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:05.206844 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e923232-087c-4cb9-80a9-a547fcf7dfeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:05.208259 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e923232-087c-4cb9-80a9-a547fcf7dfeb None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:05.208545 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e923232-087c-4cb9-80a9-a547fcf7dfeb None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:05.208822 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e923232-087c-4cb9-80a9-a547fcf7dfeb None admin] PATH_INFO: `/v3/users/099f629ee67840b9901c7ef6d7bfe89e` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:05.210001 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e923232-087c-4cb9-80a9-a547fcf7dfeb None admin] RBAC: Authorizing `identity:get_user(user_id=099f629ee67840b9901c7ef6d7bfe89e)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:05.211414 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e923232-087c-4cb9-80a9-a547fcf7dfeb None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:05.213070 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 544/1082] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Jun 10 12:46:05 2022] GET /identity/v3/users/099f629ee67840b9901c7ef6d7bfe89e => generated 285 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:46:05.218676 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4488b9-b12a-48d3-b9ab-98baa3a9969c None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:05.283555 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:46:05.318003 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4488b9-b12a-48d3-b9ab-98baa3a9969c None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:05.351015 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377f4585-852f-431f-ae70-c1e92d23d7c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:05.351894 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377f4585-852f-431f-ae70-c1e92d23d7c2 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:05.352175 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377f4585-852f-431f-ae70-c1e92d23d7c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:05.352449 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377f4585-852f-431f-ae70-c1e92d23d7c2 None admin] PATH_INFO: `/v3/projects/e9d856a2f36941c9a3e6819085041374` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:05.353623 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377f4585-852f-431f-ae70-c1e92d23d7c2 None admin] RBAC: Authorizing `identity:get_project(project_id=e9d856a2f36941c9a3e6819085041374)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:05.355046 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377f4585-852f-431f-ae70-c1e92d23d7c2 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:05.356774 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 539/1083] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Jun 10 12:46:05 2022] GET /identity/v3/projects/e9d856a2f36941c9a3e6819085041374 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:46:05.362481 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e923232-087c-4cb9-80a9-a547fcf7dfeb None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:05.445617 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:46:05.460515 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e923232-087c-4cb9-80a9-a547fcf7dfeb None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:05.492809 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66286c1-581d-4148-9016-73e1692fc24e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:05.493689 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66286c1-581d-4148-9016-73e1692fc24e None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:05.493972 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66286c1-581d-4148-9016-73e1692fc24e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:05.494247 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66286c1-581d-4148-9016-73e1692fc24e None admin] PATH_INFO: `/v3/projects/e9d856a2f36941c9a3e6819085041374/users/099f629ee67840b9901c7ef6d7bfe89e/roles/b43d5a8247e244869c2c70f986f42e8a` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:05.496207 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66286c1-581d-4148-9016-73e1692fc24e None admin] RBAC: Authorizing `identity:create_grant(project_id=e9d856a2f36941c9a3e6819085041374, user_id=099f629ee67840b9901c7ef6d7bfe89e, role_id=b43d5a8247e244869c2c70f986f42e8a)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:05.497623 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66286c1-581d-4148-9016-73e1692fc24e None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:05.576116 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 545/1084] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Jun 10 12:46:05 2022] PUT /identity/v3/projects/e9d856a2f36941c9a3e6819085041374/users/099f629ee67840b9901c7ef6d7bfe89e/roles/b43d5a8247e244869c2c70f986f42e8a => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 10 12:46:06.748624 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94180b0d-a9d6-4093-a246-49d0b2cf7fec None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:06.749151 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94180b0d-a9d6-4093-a246-49d0b2cf7fec None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:06.749151 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94180b0d-a9d6-4093-a246-49d0b2cf7fec None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:06.750531 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 540/1085] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:46:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:46:06.755781 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faf4e1f-dba1-4e9f-8b90-a341282002b6 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:06.756218 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faf4e1f-dba1-4e9f-8b90-a341282002b6 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:06.756367 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faf4e1f-dba1-4e9f-8b90-a341282002b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:06.772540 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-2faf4e1f-dba1-4e9f-8b90-a341282002b6 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:46:06.773696 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2faf4e1f-dba1-4e9f-8b90-a341282002b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:46:06.836675 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 546/1086] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:46:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 10 12:46:06.859531 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94180b0d-a9d6-4093-a246-49d0b2cf7fec None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:06.959566 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94180b0d-a9d6-4093-a246-49d0b2cf7fec None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:06.992124 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0a6738-1b49-4bae-8e26-03feeb283ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:06.992929 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0a6738-1b49-4bae-8e26-03feeb283ce5 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:06.993215 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0a6738-1b49-4bae-8e26-03feeb283ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:06.993494 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0a6738-1b49-4bae-8e26-03feeb283ce5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:06.995486 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:46:07.009243 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0a6738-1b49-4bae-8e26-03feeb283ce5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:46:07.012714 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-eb0a6738-1b49-4bae-8e26-03feeb283ce5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 10 12:46:07.012714 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:46:07.012714 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:46:07.012714 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:46:07.012714 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:46:07.012714 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:46:07.012714 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:46:07.012714 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:46:07.012714 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:46:07.012714 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:46:07.012714 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:46:07.012714 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:46:07.012714 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:46:07.012714 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:46:07.012714 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:46:07.012714 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:46:07.013637 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:46:07.013637 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:46:07.013637 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:46:07.013637 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:46:07.013637 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:46:07.013637 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:46:07.013637 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:46:07.013637 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:46:07.013637 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:46:07.013637 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:46:07.013637 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:46:07.013637 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:46:07.013637 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:46:07.013637 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:46:07.013637 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:46:07.013637 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:46:07.014742 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:46:07.014742 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:46:07.014742 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:46:07.014742 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:46:07.014742 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:46:07.014742 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:46:07.014742 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:46:07.014742 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:46:07.014742 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:46:07.014742 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:46:07.014742 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 10 12:46:07.014742 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:46:07.014742 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 541/1087] 10.222.0.27 () {56 vars in 1120 bytes} [Fri Jun 10 12:46:06 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:46:07.019912 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faf4e1f-dba1-4e9f-8b90-a341282002b6 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:07.095108 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:46:07.117664 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2faf4e1f-dba1-4e9f-8b90-a341282002b6 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:07.151179 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004b17bb-03d0-4f20-bfea-ee946f465851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:07.151845 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004b17bb-03d0-4f20-bfea-ee946f465851 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:07.152229 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004b17bb-03d0-4f20-bfea-ee946f465851 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:07.152558 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004b17bb-03d0-4f20-bfea-ee946f465851 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:07.153311 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004b17bb-03d0-4f20-bfea-ee946f465851 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:46:07.153869 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004b17bb-03d0-4f20-bfea-ee946f465851 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:07.155488 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004b17bb-03d0-4f20-bfea-ee946f465851 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:07.164711 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 547/1088] 10.222.0.27 () {56 vars in 1115 bytes} [Fri Jun 10 12:46:07 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:46:07.170372 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0a6738-1b49-4bae-8e26-03feeb283ce5 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:07.267130 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0a6738-1b49-4bae-8e26-03feeb283ce5 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:07.301419 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0661d56c-7f50-40f5-a28c-1d89be47ffe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:07.302204 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0661d56c-7f50-40f5-a28c-1d89be47ffe1 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:07.302490 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0661d56c-7f50-40f5-a28c-1d89be47ffe1 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:07.302768 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0661d56c-7f50-40f5-a28c-1d89be47ffe1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:07.314634 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0661d56c-7f50-40f5-a28c-1d89be47ffe1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:07.317414 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0661d56c-7f50-40f5-a28c-1d89be47ffe1 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:07.337236 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-0661d56c-7f50-40f5-a28c-1d89be47ffe1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:46:07.337236 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:46:07.337236 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:46:07.337236 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:46:07.337236 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:46:07.337236 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:46:07.337236 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:46:07.337236 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:46:07.337236 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:46:07.337236 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:46:07.337236 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:46:07.337236 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:46:07.337236 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 10 12:46:07.337236 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 10 12:46:07.337236 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 10 12:46:07.338331 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 10 12:46:07.338331 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:46:07.338331 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:46:07.338331 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:46:07.338331 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:46:07.338331 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:46:07.338331 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:46:07.338331 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:46:07.338331 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:46:07.338331 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:46:07.338331 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:46:07.338331 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:46:07.338331 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:46:07.338331 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:46:07.338331 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:46:07.338331 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:46:07.338331 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:46:07.339501 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 10 12:46:07.339501 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 10 12:46:07.339501 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:46:07.339501 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:46:07.340176 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 542/1089] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:46:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:46:07.344032 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004b17bb-03d0-4f20-bfea-ee946f465851 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:07.439943 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004b17bb-03d0-4f20-bfea-ee946f465851 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:07.471723 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed445eea-d521-43f8-b971-e7965303e573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:07.472401 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed445eea-d521-43f8-b971-e7965303e573 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:07.472685 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed445eea-d521-43f8-b971-e7965303e573 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:07.473009 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed445eea-d521-43f8-b971-e7965303e573 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:07.474234 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed445eea-d521-43f8-b971-e7965303e573 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:46:07.474538 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed445eea-d521-43f8-b971-e7965303e573 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:07.475905 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed445eea-d521-43f8-b971-e7965303e573 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:07.482678 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:46:07.487703 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 548/1090] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Jun 10 12:46:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:46:07.493468 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0661d56c-7f50-40f5-a28c-1d89be47ffe1 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:07.539559 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:46:07.592753 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0661d56c-7f50-40f5-a28c-1d89be47ffe1 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:07.633843 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcda7af-ba9a-40c7-a605-7fe5c8963b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:07.634632 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcda7af-ba9a-40c7-a605-7fe5c8963b12 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:07.634912 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcda7af-ba9a-40c7-a605-7fe5c8963b12 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:07.635182 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcda7af-ba9a-40c7-a605-7fe5c8963b12 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:07.642738 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcda7af-ba9a-40c7-a605-7fe5c8963b12 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:07.644194 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcda7af-ba9a-40c7-a605-7fe5c8963b12 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:07.658109 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-bdcda7af-ba9a-40c7-a605-7fe5c8963b12 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 10 12:46:07.658109 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:46:07.658109 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:46:07.658109 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:46:07.658109 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:46:07.658109 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:46:07.658109 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:46:07.658109 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:46:07.658109 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:46:07.658109 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:46:07.658109 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:46:07.658109 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:46:07.658109 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 10 12:46:07.658109 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 10 12:46:07.658109 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 10 12:46:07.658109 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 10 12:46:07.659222 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:46:07.659222 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:46:07.659222 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 10 12:46:07.659222 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 10 12:46:07.659222 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 10 12:46:07.659222 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 10 12:46:07.659222 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:46:07.659222 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:46:07.659222 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:46:07.659222 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:46:07.659222 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:46:07.659222 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:46:07.659222 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:46:07.659222 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:46:07.659222 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:46:07.659222 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:46:07.659222 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:46:07.659222 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:46:07.660650 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:46:07.660650 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:46:07.660650 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 10 12:46:07.660650 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 10 12:46:07.660650 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 10 12:46:07.660650 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 10 12:46:07.660650 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 10 12:46:07.660650 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 10 12:46:07.660650 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 10 12:46:07.660650 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:46:07.664031 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 543/1091] 10.222.0.27 () {56 vars in 1093 bytes} [Fri Jun 10 12:46:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:46:07.664535 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc4ab2e-1eb1-4aa7-8e8e-a13b0c2faad2 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:07.664834 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc4ab2e-1eb1-4aa7-8e8e-a13b0c2faad2 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:07.665105 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcda7af-ba9a-40c7-a605-7fe5c8963b12 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:07.665215 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc4ab2e-1eb1-4aa7-8e8e-a13b0c2faad2 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:07.666469 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 549/1092] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:46:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 10 12:46:07.671335 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abef6e29-07ed-4c60-b228-3ad28fd720e5 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:07.671622 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abef6e29-07ed-4c60-b228-3ad28fd720e5 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:07.671936 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abef6e29-07ed-4c60-b228-3ad28fd720e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:07.688081 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-abef6e29-07ed-4c60-b228-3ad28fd720e5 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:46:07.689266 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-abef6e29-07ed-4c60-b228-3ad28fd720e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:46:07.755153 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 550/1093] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:46:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 10 12:46:07.761718 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcda7af-ba9a-40c7-a605-7fe5c8963b12 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:07.768428 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:46:07.778763 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abef6e29-07ed-4c60-b228-3ad28fd720e5 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:07.795689 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0e2d0e-3a1f-4ffd-97e5-4249979eef50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:07.796320 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0e2d0e-3a1f-4ffd-97e5-4249979eef50 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:07.796601 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0e2d0e-3a1f-4ffd-97e5-4249979eef50 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:07.796874 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0e2d0e-3a1f-4ffd-97e5-4249979eef50 None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:07.797700 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0e2d0e-3a1f-4ffd-97e5-4249979eef50 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:46:07.798001 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0e2d0e-3a1f-4ffd-97e5-4249979eef50 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:07.799422 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0e2d0e-3a1f-4ffd-97e5-4249979eef50 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:07.816080 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 544/1094] 10.222.0.27 () {56 vars in 1133 bytes} [Fri Jun 10 12:46:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:46:07.822282 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0e2d0e-3a1f-4ffd-97e5-4249979eef50 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:07.880808 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abef6e29-07ed-4c60-b228-3ad28fd720e5 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:07.907381 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:46:07.916113 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2df5a9-deb0-4913-abe4-025091e1c246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:07.917081 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2df5a9-deb0-4913-abe4-025091e1c246 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:07.917444 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2df5a9-deb0-4913-abe4-025091e1c246 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:07.917631 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2df5a9-deb0-4913-abe4-025091e1c246 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:07.920914 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0e2d0e-3a1f-4ffd-97e5-4249979eef50 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:07.928266 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2df5a9-deb0-4913-abe4-025091e1c246 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:46:07.935816 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-5a2df5a9-deb0-4913-abe4-025091e1c246 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 10 12:46:07.935816 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:46:07.935816 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:46:07.935816 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:46:07.935816 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:46:07.935816 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:46:07.935816 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:46:07.935816 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:46:07.935816 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:46:07.935816 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:46:07.935816 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:46:07.935816 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:46:07.935816 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:46:07.935816 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:46:07.935816 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:46:07.935816 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:46:07.936830 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:46:07.936830 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:46:07.936830 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:46:07.936830 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:46:07.936830 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:46:07.936830 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:46:07.936830 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:46:07.936830 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:46:07.936830 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:46:07.936830 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:46:07.936830 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:46:07.936830 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:46:07.936830 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:46:07.936830 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:46:07.936830 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:46:07.936830 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:46:07.937688 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:46:07.937688 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:46:07.937688 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:46:07.937688 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:46:07.937688 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:46:07.937688 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:46:07.937688 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:46:07.937688 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:46:07.937688 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:46:07.937688 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:46:07.937688 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 10 12:46:07.937688 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:46:07.937688 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 551/1095] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Jun 10 12:46:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 10 12:46:07.943439 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2df5a9-deb0-4913-abe4-025091e1c246 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:07.955827 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61431963-8833-493c-8e57-09575109ebf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:07.956492 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61431963-8833-493c-8e57-09575109ebf5 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:07.956771 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61431963-8833-493c-8e57-09575109ebf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:07.957043 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61431963-8833-493c-8e57-09575109ebf5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:07.985909 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:46:08.023076 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61431963-8833-493c-8e57-09575109ebf5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:08.024551 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61431963-8833-493c-8e57-09575109ebf5 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:08.040315 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2df5a9-deb0-4913-abe4-025091e1c246 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:08.073546 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6913eadd-aba0-4d55-9cc7-64ffc3c2e1c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:08.074136 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6913eadd-aba0-4d55-9cc7-64ffc3c2e1c3 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:08.074418 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6913eadd-aba0-4d55-9cc7-64ffc3c2e1c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:08.074692 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6913eadd-aba0-4d55-9cc7-64ffc3c2e1c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:08.075426 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6913eadd-aba0-4d55-9cc7-64ffc3c2e1c3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:46:08.075730 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6913eadd-aba0-4d55-9cc7-64ffc3c2e1c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:08.077114 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6913eadd-aba0-4d55-9cc7-64ffc3c2e1c3 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:08.084934 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 552/1096] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Jun 10 12:46:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 12:46:08.093695 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6913eadd-aba0-4d55-9cc7-64ffc3c2e1c3 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:08.119289 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 545/1097] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Jun 10 12:46:07 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 10 12:46:08.124988 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61431963-8833-493c-8e57-09575109ebf5 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:08.142234 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:46:08.185352 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:46:08.191536 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6913eadd-aba0-4d55-9cc7-64ffc3c2e1c3 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:08.222087 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61431963-8833-493c-8e57-09575109ebf5 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:08.224575 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c51f474-603b-4136-95c8-b101f56b1d29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:08.225446 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c51f474-603b-4136-95c8-b101f56b1d29 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:08.225729 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c51f474-603b-4136-95c8-b101f56b1d29 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:08.226004 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c51f474-603b-4136-95c8-b101f56b1d29 None admin] PATH_INFO: `/v3/users/099f629ee67840b9901c7ef6d7bfe89e` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:08.227200 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c51f474-603b-4136-95c8-b101f56b1d29 None admin] RBAC: Authorizing `identity:get_user(user_id=099f629ee67840b9901c7ef6d7bfe89e)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:08.228657 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c51f474-603b-4136-95c8-b101f56b1d29 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:08.230288 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 553/1098] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Jun 10 12:46:08 2022] GET /identity/v3/users/099f629ee67840b9901c7ef6d7bfe89e => generated 285 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 12:46:08.235810 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c51f474-603b-4136-95c8-b101f56b1d29 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:08.256440 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806d9198-2da0-4054-95ca-38af7145ade7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:08.257223 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806d9198-2da0-4054-95ca-38af7145ade7 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:08.257503 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806d9198-2da0-4054-95ca-38af7145ade7 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:08.257782 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806d9198-2da0-4054-95ca-38af7145ade7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:08.268043 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806d9198-2da0-4054-95ca-38af7145ade7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:08.269472 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806d9198-2da0-4054-95ca-38af7145ade7 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:08.286628 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-806d9198-2da0-4054-95ca-38af7145ade7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:46:08.286628 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:46:08.286628 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:46:08.286628 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:46:08.286628 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:46:08.286628 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:46:08.286628 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:46:08.286628 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:46:08.286628 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:46:08.286628 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:46:08.286628 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:46:08.286628 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:46:08.286628 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 10 12:46:08.286628 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 10 12:46:08.286628 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 10 12:46:08.287558 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 10 12:46:08.287558 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:46:08.287558 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:46:08.287558 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:46:08.287558 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:46:08.287558 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:46:08.287558 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:46:08.287558 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:46:08.287558 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:46:08.287558 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:46:08.287558 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:46:08.287558 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:46:08.287558 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:46:08.287558 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:46:08.287558 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:46:08.287558 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:46:08.287558 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:46:08.288847 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 10 12:46:08.288847 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 10 12:46:08.288847 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:46:08.288847 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:46:08.288847 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 546/1099] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:46:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 10 12:46:08.293330 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806d9198-2da0-4054-95ca-38af7145ade7 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:08.336163 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c51f474-603b-4136-95c8-b101f56b1d29 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:08.352592 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:46:08.369575 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17fe33e-dfe0-406a-a268-b199cdcac935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:08.370188 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17fe33e-dfe0-406a-a268-b199cdcac935 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:08.370470 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17fe33e-dfe0-406a-a268-b199cdcac935 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:08.370748 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17fe33e-dfe0-406a-a268-b199cdcac935 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:08.388912 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806d9198-2da0-4054-95ca-38af7145ade7 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:08.404283 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:46:08.421879 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938c9268-0863-4fd1-b7c8-22d71d75aa2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:08.422529 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938c9268-0863-4fd1-b7c8-22d71d75aa2f None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:08.422813 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938c9268-0863-4fd1-b7c8-22d71d75aa2f None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:08.423092 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938c9268-0863-4fd1-b7c8-22d71d75aa2f None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:08.423807 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938c9268-0863-4fd1-b7c8-22d71d75aa2f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:46:08.424148 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938c9268-0863-4fd1-b7c8-22d71d75aa2f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:08.425528 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938c9268-0863-4fd1-b7c8-22d71d75aa2f None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:08.437578 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17fe33e-dfe0-406a-a268-b199cdcac935 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:08.438106 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 547/1100] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Jun 10 12:46:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 12:46:08.439000 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17fe33e-dfe0-406a-a268-b199cdcac935 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:08.443270 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938c9268-0863-4fd1-b7c8-22d71d75aa2f None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:08.534900 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 554/1101] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Jun 10 12:46:08 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 10 12:46:08.539398 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938c9268-0863-4fd1-b7c8-22d71d75aa2f None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:08.541013 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17fe33e-dfe0-406a-a268-b199cdcac935 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:08.557609 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:46:08.573518 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492633fe-7e9c-4885-bbba-a65aaef2fd19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:08.576759 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492633fe-7e9c-4885-bbba-a65aaef2fd19 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:08.576759 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492633fe-7e9c-4885-bbba-a65aaef2fd19 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:08.576759 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492633fe-7e9c-4885-bbba-a65aaef2fd19 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:08.581312 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492633fe-7e9c-4885-bbba-a65aaef2fd19 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:08.582796 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492633fe-7e9c-4885-bbba-a65aaef2fd19 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:08.596753 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-492633fe-7e9c-4885-bbba-a65aaef2fd19 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 10 12:46:08.596753 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:46:08.596753 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:46:08.596753 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:46:08.596753 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:46:08.596753 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:46:08.596753 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:46:08.596753 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:46:08.596753 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:46:08.596753 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:46:08.596753 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:46:08.596753 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:46:08.596753 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 10 12:46:08.596753 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 10 12:46:08.596753 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 10 12:46:08.597658 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 10 12:46:08.597658 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:46:08.597658 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:46:08.597658 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:46:08.597658 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:46:08.597658 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:46:08.597658 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:46:08.597658 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:46:08.597658 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:46:08.597658 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:46:08.597658 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:46:08.597658 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:46:08.597658 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:46:08.597658 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:46:08.597658 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:46:08.597658 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:46:08.597658 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:46:08.598632 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 10 12:46:08.598632 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 10 12:46:08.598632 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 10 12:46:08.598632 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 10 12:46:08.598632 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 10 12:46:08.598632 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 10 12:46:08.598632 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 10 12:46:08.598632 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:46:08.598632 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 548/1102] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Jun 10 12:46:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 10 12:46:08.603476 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492633fe-7e9c-4885-bbba-a65aaef2fd19 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:08.637464 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17fe33e-dfe0-406a-a268-b199cdcac935 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:08.639402 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:46:08.670055 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2652c9-e023-46f7-b4f6-d8da34a6110f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:08.670905 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2652c9-e023-46f7-b4f6-d8da34a6110f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:08.671184 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2652c9-e023-46f7-b4f6-d8da34a6110f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:08.671460 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2652c9-e023-46f7-b4f6-d8da34a6110f None admin] PATH_INFO: `/v3/projects/e9d856a2f36941c9a3e6819085041374` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:08.672738 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2652c9-e023-46f7-b4f6-d8da34a6110f None admin] RBAC: Authorizing `identity:get_project(project_id=e9d856a2f36941c9a3e6819085041374)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:08.674136 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2652c9-e023-46f7-b4f6-d8da34a6110f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:08.675812 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 555/1103] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Jun 10 12:46:08 2022] GET /identity/v3/projects/e9d856a2f36941c9a3e6819085041374 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:46:08.682393 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2652c9-e023-46f7-b4f6-d8da34a6110f None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:08.703252 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492633fe-7e9c-4885-bbba-a65aaef2fd19 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:08.735563 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5d23e5-6e6c-4cda-8ce6-04fb2fee6279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:08.736251 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5d23e5-6e6c-4cda-8ce6-04fb2fee6279 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:08.736529 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5d23e5-6e6c-4cda-8ce6-04fb2fee6279 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:08.736802 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5d23e5-6e6c-4cda-8ce6-04fb2fee6279 None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:08.737581 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5d23e5-6e6c-4cda-8ce6-04fb2fee6279 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:46:08.737885 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5d23e5-6e6c-4cda-8ce6-04fb2fee6279 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:08.739840 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5d23e5-6e6c-4cda-8ce6-04fb2fee6279 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:08.751486 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 549/1104] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Jun 10 12:46:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 12:46:08.757918 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5d23e5-6e6c-4cda-8ce6-04fb2fee6279 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:08.766712 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:46:08.781736 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2652c9-e023-46f7-b4f6-d8da34a6110f None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:08.815260 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7d26ae-78d9-4a93-9126-c607c25b2efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:08.815924 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7d26ae-78d9-4a93-9126-c607c25b2efa None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:08.816207 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7d26ae-78d9-4a93-9126-c607c25b2efa None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:08.816488 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7d26ae-78d9-4a93-9126-c607c25b2efa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:08.817911 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7d26ae-78d9-4a93-9126-c607c25b2efa None admin] RBAC: Adding query filter params (role.id=b43d5a8247e244869c2c70f986f42e8a, scope.project.id=e9d856a2f36941c9a3e6819085041374, user.id=099f629ee67840b9901c7ef6d7bfe89e) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:46:08.818223 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7d26ae-78d9-4a93-9126-c607c25b2efa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:08.819622 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7d26ae-78d9-4a93-9126-c607c25b2efa None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:08.825292 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 556/1105] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Jun 10 12:46:08 2022] GET /identity/v3/role_assignments?user.id=099f629ee67840b9901c7ef6d7bfe89e&scope.project.id=e9d856a2f36941c9a3e6819085041374&role.id=b43d5a8247e244869c2c70f986f42e8a => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 12:46:08.852568 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:46:08.860001 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5d23e5-6e6c-4cda-8ce6-04fb2fee6279 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:08.897926 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fa82e9-1e9e-426d-90a4-d3c9123c157b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:08.898598 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fa82e9-1e9e-426d-90a4-d3c9123c157b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:08.898880 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fa82e9-1e9e-426d-90a4-d3c9123c157b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:08.899158 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fa82e9-1e9e-426d-90a4-d3c9123c157b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:08.900050 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fa82e9-1e9e-426d-90a4-d3c9123c157b None admin] RBAC: Adding query filter params (role.id=55ae4712a75e4f9bb542575264d55f64, scope.project.id=73ea5b23a2d34544ab493f4c5e4034b8, user.id=8bd6caa461e84100a3d5fe5af5d80f3e) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:46:08.900365 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fa82e9-1e9e-426d-90a4-d3c9123c157b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:08.901834 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fa82e9-1e9e-426d-90a4-d3c9123c157b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:08.908603 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 550/1106] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Jun 10 12:46:08 2022] GET /identity/v3/role_assignments?user.id=8bd6caa461e84100a3d5fe5af5d80f3e&scope.project.id=73ea5b23a2d34544ab493f4c5e4034b8&role.id=55ae4712a75e4f9bb542575264d55f64 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 12:46:10.942105 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faed823-3fa8-4f9c-a180-77da64350bec None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:10.942653 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faed823-3fa8-4f9c-a180-77da64350bec None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:10.942653 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faed823-3fa8-4f9c-a180-77da64350bec None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:10.944094 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 557/1107] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:46:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:46:10.949165 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad65c37-e47c-4783-9a71-7f63822b8792 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:10.949576 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad65c37-e47c-4783-9a71-7f63822b8792 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:10.949729 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad65c37-e47c-4783-9a71-7f63822b8792 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:10.966437 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-fad65c37-e47c-4783-9a71-7f63822b8792 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:46:10.967604 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-fad65c37-e47c-4783-9a71-7f63822b8792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:46:11.031420 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 551/1108] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:46:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 10 12:46:11.055953 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faed823-3fa8-4f9c-a180-77da64350bec None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:11.152222 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faed823-3fa8-4f9c-a180-77da64350bec None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:11.184490 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ac6338-2f7c-461b-af98-eb265a492ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:11.185258 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ac6338-2f7c-461b-af98-eb265a492ed7 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:11.185538 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ac6338-2f7c-461b-af98-eb265a492ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:11.185809 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ac6338-2f7c-461b-af98-eb265a492ed7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:11.186954 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ac6338-2f7c-461b-af98-eb265a492ed7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:11.188395 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ac6338-2f7c-461b-af98-eb265a492ed7 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:11.189967 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 558/1109] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:46:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:46:11.195853 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad65c37-e47c-4783-9a71-7f63822b8792 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:11.285682 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:46:11.294555 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad65c37-e47c-4783-9a71-7f63822b8792 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:11.327021 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b6b137-ac95-4ac5-8730-2b75ca0d0bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:11.327680 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b6b137-ac95-4ac5-8730-2b75ca0d0bd8 None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:11.328063 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b6b137-ac95-4ac5-8730-2b75ca0d0bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:11.328343 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b6b137-ac95-4ac5-8730-2b75ca0d0bd8 None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:11.329050 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b6b137-ac95-4ac5-8730-2b75ca0d0bd8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:11.330484 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b6b137-ac95-4ac5-8730-2b75ca0d0bd8 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:11.410595 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 552/1110] 10.222.0.27 () {60 vars in 1132 bytes} [Fri Jun 10 12:46:11 2022] POST /identity/v3/users => generated 286 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 12:46:13.500814 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46382aa1-6da7-4823-9b08-0fc9ffbbf6d0 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:13.501384 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46382aa1-6da7-4823-9b08-0fc9ffbbf6d0 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:13.501384 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46382aa1-6da7-4823-9b08-0fc9ffbbf6d0 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:13.502711 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 559/1111] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:46:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:46:13.508386 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87a1fcc-219f-4b96-9718-2556fa0b43f0 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:13.508755 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87a1fcc-219f-4b96-9718-2556fa0b43f0 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:13.508955 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87a1fcc-219f-4b96-9718-2556fa0b43f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:13.525859 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-e87a1fcc-219f-4b96-9718-2556fa0b43f0 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:46:13.527031 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e87a1fcc-219f-4b96-9718-2556fa0b43f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:46:13.590783 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 553/1112] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:46:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 10 12:46:13.613440 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46382aa1-6da7-4823-9b08-0fc9ffbbf6d0 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:13.632308 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:46:13.710960 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46382aa1-6da7-4823-9b08-0fc9ffbbf6d0 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:13.742411 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7fe3a8-dc22-44fe-bcbf-44e98a1df533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:13.743243 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7fe3a8-dc22-44fe-bcbf-44e98a1df533 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:13.743522 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7fe3a8-dc22-44fe-bcbf-44e98a1df533 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:13.743795 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7fe3a8-dc22-44fe-bcbf-44e98a1df533 None admin] PATH_INFO: `/v3/roles/2d71ee7c2a544fe5ad77d92b24280efe` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:13.745258 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7fe3a8-dc22-44fe-bcbf-44e98a1df533 None admin] RBAC: Authorizing `identity:get_role(role_id=2d71ee7c2a544fe5ad77d92b24280efe)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:13.746599 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7fe3a8-dc22-44fe-bcbf-44e98a1df533 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:13.747951 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 560/1113] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Jun 10 12:46:13 2022] GET /identity/v3/roles/2d71ee7c2a544fe5ad77d92b24280efe => generated 223 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:46:13.753475 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87a1fcc-219f-4b96-9718-2556fa0b43f0 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:13.827049 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:46:13.849000 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87a1fcc-219f-4b96-9718-2556fa0b43f0 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:13.883101 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17eebe4-48ba-4fd3-a4fb-9411c8830df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:13.884051 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17eebe4-48ba-4fd3-a4fb-9411c8830df9 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:13.884528 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17eebe4-48ba-4fd3-a4fb-9411c8830df9 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:13.885013 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17eebe4-48ba-4fd3-a4fb-9411c8830df9 None admin] PATH_INFO: `/v3/users/0850e1bf2059472faa39fbd196b83127` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:13.898165 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17eebe4-48ba-4fd3-a4fb-9411c8830df9 None admin] RBAC: Authorizing `identity:get_user(user_id=0850e1bf2059472faa39fbd196b83127)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:13.899608 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17eebe4-48ba-4fd3-a4fb-9411c8830df9 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:13.901266 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 554/1114] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Jun 10 12:46:13 2022] GET /identity/v3/users/0850e1bf2059472faa39fbd196b83127 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:46:13.906988 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7fe3a8-dc22-44fe-bcbf-44e98a1df533 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:13.987881 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:46:14.001914 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7fe3a8-dc22-44fe-bcbf-44e98a1df533 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:14.034112 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaf2883-18d3-43e7-b64e-f01f3f367dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:14.034681 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaf2883-18d3-43e7-b64e-f01f3f367dc3 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:14.034963 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaf2883-18d3-43e7-b64e-f01f3f367dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:14.035529 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaf2883-18d3-43e7-b64e-f01f3f367dc3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:14.098521 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaf2883-18d3-43e7-b64e-f01f3f367dc3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:14.099954 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaf2883-18d3-43e7-b64e-f01f3f367dc3 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:14.178153 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:46:14.192862 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 561/1115] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Jun 10 12:46:13 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 10 12:46:14.198667 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17eebe4-48ba-4fd3-a4fb-9411c8830df9 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:14.299837 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17eebe4-48ba-4fd3-a4fb-9411c8830df9 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:14.332358 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe8c51a-0612-4238-a559-926a7a570e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:14.333197 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe8c51a-0612-4238-a559-926a7a570e9b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:14.333481 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe8c51a-0612-4238-a559-926a7a570e9b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:14.333757 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe8c51a-0612-4238-a559-926a7a570e9b None admin] PATH_INFO: `/v3/projects/e9d856a2f36941c9a3e6819085041374` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:14.334910 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe8c51a-0612-4238-a559-926a7a570e9b None admin] RBAC: Authorizing `identity:get_project(project_id=e9d856a2f36941c9a3e6819085041374)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:14.336380 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe8c51a-0612-4238-a559-926a7a570e9b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:14.338033 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 555/1116] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Jun 10 12:46:14 2022] GET /identity/v3/projects/e9d856a2f36941c9a3e6819085041374 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:46:14.343961 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaf2883-18d3-43e7-b64e-f01f3f367dc3 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:14.438296 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaf2883-18d3-43e7-b64e-f01f3f367dc3 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:14.470254 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09049f9-5cda-48b3-903e-85c2f43da7c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:14.470887 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09049f9-5cda-48b3-903e-85c2f43da7c7 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:14.471169 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09049f9-5cda-48b3-903e-85c2f43da7c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:14.471444 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09049f9-5cda-48b3-903e-85c2f43da7c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:14.472290 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09049f9-5cda-48b3-903e-85c2f43da7c7 None admin] RBAC: Adding query filter params (role.id=2d71ee7c2a544fe5ad77d92b24280efe, scope.project.id=e9d856a2f36941c9a3e6819085041374, user.id=0850e1bf2059472faa39fbd196b83127) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:46:14.473137 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09049f9-5cda-48b3-903e-85c2f43da7c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:14.474537 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09049f9-5cda-48b3-903e-85c2f43da7c7 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:14.479764 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 562/1117] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Jun 10 12:46:14 2022] GET /identity/v3/role_assignments?user.id=0850e1bf2059472faa39fbd196b83127&scope.project.id=e9d856a2f36941c9a3e6819085041374&role.id=2d71ee7c2a544fe5ad77d92b24280efe => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:46:16.536443 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde2b20d-d38d-4ffe-8747-64c0766238d2 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:16.537069 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde2b20d-d38d-4ffe-8747-64c0766238d2 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:16.537069 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde2b20d-d38d-4ffe-8747-64c0766238d2 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:16.538347 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 556/1118] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:46:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:46:16.543594 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2be5fb3-a3cb-47b0-8eda-b4c1c6f2f73b None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:16.544075 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2be5fb3-a3cb-47b0-8eda-b4c1c6f2f73b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:16.544200 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2be5fb3-a3cb-47b0-8eda-b4c1c6f2f73b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:16.560326 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-b2be5fb3-a3cb-47b0-8eda-b4c1c6f2f73b None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:46:16.561492 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b2be5fb3-a3cb-47b0-8eda-b4c1c6f2f73b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:46:16.606128 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:46:16.626004 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 563/1119] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:46:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 10 12:46:16.649412 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde2b20d-d38d-4ffe-8747-64c0766238d2 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:16.652256 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:46:16.746767 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde2b20d-d38d-4ffe-8747-64c0766238d2 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:16.779511 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb3e30e-9eec-4b8d-9f2b-31627f1c7c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:16.780367 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb3e30e-9eec-4b8d-9f2b-31627f1c7c53 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:16.780649 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb3e30e-9eec-4b8d-9f2b-31627f1c7c53 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:16.780923 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb3e30e-9eec-4b8d-9f2b-31627f1c7c53 None admin] PATH_INFO: `/v3/roles/2d71ee7c2a544fe5ad77d92b24280efe` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:16.782358 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb3e30e-9eec-4b8d-9f2b-31627f1c7c53 None admin] RBAC: Authorizing `identity:get_role(role_id=2d71ee7c2a544fe5ad77d92b24280efe)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:16.783703 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb3e30e-9eec-4b8d-9f2b-31627f1c7c53 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:16.785335 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 557/1120] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Jun 10 12:46:16 2022] GET /identity/v3/roles/2d71ee7c2a544fe5ad77d92b24280efe => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:46:16.791061 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2be5fb3-a3cb-47b0-8eda-b4c1c6f2f73b None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:16.889166 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2be5fb3-a3cb-47b0-8eda-b4c1c6f2f73b None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:16.922650 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf791bb7-9dec-4a05-9584-77e75709faef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:16.923502 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf791bb7-9dec-4a05-9584-77e75709faef None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:16.923788 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf791bb7-9dec-4a05-9584-77e75709faef None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:16.924095 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf791bb7-9dec-4a05-9584-77e75709faef None admin] PATH_INFO: `/v3/users/0850e1bf2059472faa39fbd196b83127` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:16.925259 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf791bb7-9dec-4a05-9584-77e75709faef None admin] RBAC: Authorizing `identity:get_user(user_id=0850e1bf2059472faa39fbd196b83127)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:16.927226 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf791bb7-9dec-4a05-9584-77e75709faef None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:16.928933 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 564/1121] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Jun 10 12:46:16 2022] GET /identity/v3/users/0850e1bf2059472faa39fbd196b83127 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:46:16.934770 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb3e30e-9eec-4b8d-9f2b-31627f1c7c53 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:17.006869 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:46:17.035410 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb3e30e-9eec-4b8d-9f2b-31627f1c7c53 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:17.069197 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd801a0a-7a4f-402b-9177-31f6cefbd082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:17.069979 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd801a0a-7a4f-402b-9177-31f6cefbd082 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:17.070264 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd801a0a-7a4f-402b-9177-31f6cefbd082 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:17.070536 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd801a0a-7a4f-402b-9177-31f6cefbd082 None admin] PATH_INFO: `/v3/projects/e9d856a2f36941c9a3e6819085041374` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:17.071689 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd801a0a-7a4f-402b-9177-31f6cefbd082 None admin] RBAC: Authorizing `identity:get_project(project_id=e9d856a2f36941c9a3e6819085041374)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:17.073129 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd801a0a-7a4f-402b-9177-31f6cefbd082 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:17.074794 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 558/1122] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Jun 10 12:46:16 2022] GET /identity/v3/projects/e9d856a2f36941c9a3e6819085041374 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:46:17.080677 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf791bb7-9dec-4a05-9584-77e75709faef None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:17.127532 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:46:17.176047 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf791bb7-9dec-4a05-9584-77e75709faef None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:17.208773 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e25ee5-d801-441a-aecc-7d2c6efadf7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:17.209639 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e25ee5-d801-441a-aecc-7d2c6efadf7f None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:17.209920 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e25ee5-d801-441a-aecc-7d2c6efadf7f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:17.210190 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e25ee5-d801-441a-aecc-7d2c6efadf7f None admin] PATH_INFO: `/v3/projects/e9d856a2f36941c9a3e6819085041374/users/0850e1bf2059472faa39fbd196b83127/roles/2d71ee7c2a544fe5ad77d92b24280efe` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:17.212360 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e25ee5-d801-441a-aecc-7d2c6efadf7f None admin] RBAC: Authorizing `identity:create_grant(project_id=e9d856a2f36941c9a3e6819085041374, user_id=0850e1bf2059472faa39fbd196b83127, role_id=2d71ee7c2a544fe5ad77d92b24280efe)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:17.213759 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e25ee5-d801-441a-aecc-7d2c6efadf7f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:17.261261 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 565/1123] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Jun 10 12:46:17 2022] PUT /identity/v3/projects/e9d856a2f36941c9a3e6819085041374/users/0850e1bf2059472faa39fbd196b83127/roles/2d71ee7c2a544fe5ad77d92b24280efe => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 12:46:19.355069 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e05edc-e56c-41c8-b9d0-9ed6ff721437 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:19.355586 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e05edc-e56c-41c8-b9d0-9ed6ff721437 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:19.355586 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e05edc-e56c-41c8-b9d0-9ed6ff721437 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:19.356932 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 559/1124] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:46:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:46:19.361917 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7038b0-6364-44c9-aa37-569a3293b7ba None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:19.362288 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7038b0-6364-44c9-aa37-569a3293b7ba None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:19.362479 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7038b0-6364-44c9-aa37-569a3293b7ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:19.379392 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-5c7038b0-6364-44c9-aa37-569a3293b7ba None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:46:19.380597 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5c7038b0-6364-44c9-aa37-569a3293b7ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:46:19.444421 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 566/1125] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:46:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 10 12:46:19.467286 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e05edc-e56c-41c8-b9d0-9ed6ff721437 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:19.565859 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e05edc-e56c-41c8-b9d0-9ed6ff721437 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:19.598588 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fb5bb8-0b6d-4d79-9a09-fa4d0460f7aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:19.599424 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fb5bb8-0b6d-4d79-9a09-fa4d0460f7aa None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:19.599704 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fb5bb8-0b6d-4d79-9a09-fa4d0460f7aa None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:19.600007 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fb5bb8-0b6d-4d79-9a09-fa4d0460f7aa None admin] PATH_INFO: `/v3/roles/2d71ee7c2a544fe5ad77d92b24280efe` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:19.601460 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fb5bb8-0b6d-4d79-9a09-fa4d0460f7aa None admin] RBAC: Authorizing `identity:get_role(role_id=2d71ee7c2a544fe5ad77d92b24280efe)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:19.602821 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fb5bb8-0b6d-4d79-9a09-fa4d0460f7aa None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:19.604207 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 560/1126] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Jun 10 12:46:19 2022] GET /identity/v3/roles/2d71ee7c2a544fe5ad77d92b24280efe => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:46:19.609807 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7038b0-6364-44c9-aa37-569a3293b7ba None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:19.647624 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:46:19.705464 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7038b0-6364-44c9-aa37-569a3293b7ba None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:19.738664 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0985ef-5862-4bf7-a040-92ef6608fdb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:19.739521 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0985ef-5862-4bf7-a040-92ef6608fdb0 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:19.739801 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0985ef-5862-4bf7-a040-92ef6608fdb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:19.740100 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0985ef-5862-4bf7-a040-92ef6608fdb0 None admin] PATH_INFO: `/v3/users/0850e1bf2059472faa39fbd196b83127` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:19.741265 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0985ef-5862-4bf7-a040-92ef6608fdb0 None admin] RBAC: Authorizing `identity:get_user(user_id=0850e1bf2059472faa39fbd196b83127)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:19.742699 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0985ef-5862-4bf7-a040-92ef6608fdb0 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:19.744328 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 567/1127] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Jun 10 12:46:19 2022] GET /identity/v3/users/0850e1bf2059472faa39fbd196b83127 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:46:19.749859 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fb5bb8-0b6d-4d79-9a09-fa4d0460f7aa None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:19.758388 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:46:19.846705 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fb5bb8-0b6d-4d79-9a09-fa4d0460f7aa None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:19.879737 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f948f62-1b8c-40b4-91c7-a066aa30d213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:19.880413 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f948f62-1b8c-40b4-91c7-a066aa30d213 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:19.880699 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f948f62-1b8c-40b4-91c7-a066aa30d213 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:19.880974 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f948f62-1b8c-40b4-91c7-a066aa30d213 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:19.945177 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f948f62-1b8c-40b4-91c7-a066aa30d213 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:19.946620 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f948f62-1b8c-40b4-91c7-a066aa30d213 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:19.953879 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:46:20.041235 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 561/1128] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Jun 10 12:46:19 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 10 12:46:20.047058 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0985ef-5862-4bf7-a040-92ef6608fdb0 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:20.142009 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0985ef-5862-4bf7-a040-92ef6608fdb0 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:20.149014 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:46:20.174854 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaf04e2-87a1-4411-9930-d1f4f9a7e66a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:20.175726 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaf04e2-87a1-4411-9930-d1f4f9a7e66a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:20.176040 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaf04e2-87a1-4411-9930-d1f4f9a7e66a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:20.176320 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaf04e2-87a1-4411-9930-d1f4f9a7e66a None admin] PATH_INFO: `/v3/projects/e9d856a2f36941c9a3e6819085041374` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:20.177484 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaf04e2-87a1-4411-9930-d1f4f9a7e66a None admin] RBAC: Authorizing `identity:get_project(project_id=e9d856a2f36941c9a3e6819085041374)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:20.178899 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaf04e2-87a1-4411-9930-d1f4f9a7e66a None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:20.181427 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 568/1129] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Jun 10 12:46:20 2022] GET /identity/v3/projects/e9d856a2f36941c9a3e6819085041374 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:46:20.186614 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f948f62-1b8c-40b4-91c7-a066aa30d213 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:20.281877 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f948f62-1b8c-40b4-91c7-a066aa30d213 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:20.289150 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:46:20.314100 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b79492-fe2e-4278-ad52-06d7ae18f3f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:20.314740 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b79492-fe2e-4278-ad52-06d7ae18f3f8 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:20.315024 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b79492-fe2e-4278-ad52-06d7ae18f3f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:20.315301 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b79492-fe2e-4278-ad52-06d7ae18f3f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:20.316681 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b79492-fe2e-4278-ad52-06d7ae18f3f8 None admin] RBAC: Adding query filter params (role.id=2d71ee7c2a544fe5ad77d92b24280efe, scope.project.id=e9d856a2f36941c9a3e6819085041374, user.id=0850e1bf2059472faa39fbd196b83127) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:46:20.316993 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b79492-fe2e-4278-ad52-06d7ae18f3f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:20.318557 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b79492-fe2e-4278-ad52-06d7ae18f3f8 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:20.324743 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 562/1130] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Jun 10 12:46:20 2022] GET /identity/v3/role_assignments?user.id=0850e1bf2059472faa39fbd196b83127&scope.project.id=e9d856a2f36941c9a3e6819085041374&role.id=2d71ee7c2a544fe5ad77d92b24280efe => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 12:46:22.404870 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e59ca3-c75b-4dfa-aa8b-c8625da379f1 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:22.405419 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e59ca3-c75b-4dfa-aa8b-c8625da379f1 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:22.405419 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e59ca3-c75b-4dfa-aa8b-c8625da379f1 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:22.406794 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 569/1131] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:46:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:46:22.412733 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c155cac8-d007-457f-a4bf-597835f2f608 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:22.413120 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c155cac8-d007-457f-a4bf-597835f2f608 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:22.413288 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c155cac8-d007-457f-a4bf-597835f2f608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:22.429479 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-c155cac8-d007-457f-a4bf-597835f2f608 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:46:22.430638 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c155cac8-d007-457f-a4bf-597835f2f608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:46:22.494039 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 563/1132] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:46:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 10 12:46:22.516684 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e59ca3-c75b-4dfa-aa8b-c8625da379f1 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:22.613186 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e59ca3-c75b-4dfa-aa8b-c8625da379f1 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:22.645278 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cb0334-712a-4a20-9c5a-ad6ad3d5a1e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:22.646043 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cb0334-712a-4a20-9c5a-ad6ad3d5a1e7 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:22.646323 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cb0334-712a-4a20-9c5a-ad6ad3d5a1e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:22.646595 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cb0334-712a-4a20-9c5a-ad6ad3d5a1e7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:22.647739 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cb0334-712a-4a20-9c5a-ad6ad3d5a1e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:22.649169 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cb0334-712a-4a20-9c5a-ad6ad3d5a1e7 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:22.650726 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 570/1133] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:46:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:46:22.656209 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c155cac8-d007-457f-a4bf-597835f2f608 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:22.749632 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:46:22.752462 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c155cac8-d007-457f-a4bf-597835f2f608 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:22.785866 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af2c324-019a-4a17-a6a6-ba5ea817de8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:22.786515 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af2c324-019a-4a17-a6a6-ba5ea817de8f None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:22.786797 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af2c324-019a-4a17-a6a6-ba5ea817de8f None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:22.787072 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af2c324-019a-4a17-a6a6-ba5ea817de8f None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:22.787767 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af2c324-019a-4a17-a6a6-ba5ea817de8f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:22.789228 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af2c324-019a-4a17-a6a6-ba5ea817de8f None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:22.874103 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 564/1134] 10.222.0.27 () {60 vars in 1141 bytes} [Fri Jun 10 12:46:22 2022] POST /identity/v3/projects => generated 311 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 12:46:24.974676 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef29ab7-ad7b-455e-a039-9221a4bb6df1 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:24.975240 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef29ab7-ad7b-455e-a039-9221a4bb6df1 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:24.975240 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef29ab7-ad7b-455e-a039-9221a4bb6df1 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:24.976646 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 571/1135] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:46:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:46:24.981621 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f8203c-b1df-4ef1-b638-3affad1d5386 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:24.982033 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f8203c-b1df-4ef1-b638-3affad1d5386 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:24.982183 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f8203c-b1df-4ef1-b638-3affad1d5386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:24.999114 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-69f8203c-b1df-4ef1-b638-3affad1d5386 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:46:25.000343 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-69f8203c-b1df-4ef1-b638-3affad1d5386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:46:25.068174 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 565/1136] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:46:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 10 12:46:25.092596 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef29ab7-ad7b-455e-a039-9221a4bb6df1 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:25.135534 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:46:25.191440 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef29ab7-ad7b-455e-a039-9221a4bb6df1 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:25.223988 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c85bb94-260c-4f88-a0b0-9c0c601444fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:25.224775 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c85bb94-260c-4f88-a0b0-9c0c601444fd None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:25.225064 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c85bb94-260c-4f88-a0b0-9c0c601444fd None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:25.225340 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c85bb94-260c-4f88-a0b0-9c0c601444fd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:25.226507 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c85bb94-260c-4f88-a0b0-9c0c601444fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:25.227953 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c85bb94-260c-4f88-a0b0-9c0c601444fd None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:25.229546 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 572/1137] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:46:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:46:25.234977 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f8203c-b1df-4ef1-b638-3affad1d5386 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:25.312966 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:46:25.330165 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f8203c-b1df-4ef1-b638-3affad1d5386 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:25.372013 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3199ab20-ffd4-41f3-ac4b-41eaee7ebc9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:25.372670 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3199ab20-ffd4-41f3-ac4b-41eaee7ebc9a None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:25.372951 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3199ab20-ffd4-41f3-ac4b-41eaee7ebc9a None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:25.373221 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3199ab20-ffd4-41f3-ac4b-41eaee7ebc9a None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:25.373914 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3199ab20-ffd4-41f3-ac4b-41eaee7ebc9a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:25.375333 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3199ab20-ffd4-41f3-ac4b-41eaee7ebc9a None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:25.479021 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 566/1138] 10.222.0.27 () {60 vars in 1132 bytes} [Fri Jun 10 12:46:25 2022] POST /identity/v3/users => generated 286 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 12:46:27.607926 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3ad938-cc69-4a10-a058-7ee05b519635 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:27.608490 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3ad938-cc69-4a10-a058-7ee05b519635 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:27.608490 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3ad938-cc69-4a10-a058-7ee05b519635 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:27.609820 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 573/1139] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:46:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:46:27.615368 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d5af57-b00f-4066-9b06-c10ffb56a86c None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:27.615750 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d5af57-b00f-4066-9b06-c10ffb56a86c None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:27.615959 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d5af57-b00f-4066-9b06-c10ffb56a86c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:27.632764 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-a6d5af57-b00f-4066-9b06-c10ffb56a86c None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:46:27.633935 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a6d5af57-b00f-4066-9b06-c10ffb56a86c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:46:27.697833 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 567/1140] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:46:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 10 12:46:27.720193 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3ad938-cc69-4a10-a058-7ee05b519635 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:27.817320 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3ad938-cc69-4a10-a058-7ee05b519635 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:27.826953 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:46:27.849211 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3391826c-c92c-4ace-a8d1-0aa53429f645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:27.849982 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3391826c-c92c-4ace-a8d1-0aa53429f645 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:27.850266 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3391826c-c92c-4ace-a8d1-0aa53429f645 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:27.850540 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3391826c-c92c-4ace-a8d1-0aa53429f645 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:27.861707 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3391826c-c92c-4ace-a8d1-0aa53429f645 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:46:27.869976 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-3391826c-c92c-4ace-a8d1-0aa53429f645 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 10 12:46:27.869976 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:46:27.869976 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:46:27.869976 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:46:27.869976 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:46:27.869976 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:46:27.869976 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:46:27.869976 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:46:27.869976 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:46:27.869976 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:46:27.869976 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:46:27.869976 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:46:27.869976 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:46:27.869976 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:46:27.869976 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:46:27.869976 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:46:27.871019 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:46:27.871019 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:46:27.871019 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:46:27.871019 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:46:27.871019 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:46:27.871019 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:46:27.871019 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:46:27.871019 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:46:27.871019 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:46:27.871019 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:46:27.871019 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:46:27.871019 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:46:27.871019 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:46:27.871019 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:46:27.871019 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:46:27.871019 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:46:27.871985 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:46:27.871985 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:46:27.871985 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:46:27.871985 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:46:27.871985 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:46:27.871985 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:46:27.871985 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:46:27.871985 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:46:27.871985 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:46:27.871985 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:46:27.871985 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 10 12:46:27.871985 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:46:27.871985 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 574/1141] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Jun 10 12:46:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:46:27.877207 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d5af57-b00f-4066-9b06-c10ffb56a86c None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:27.941559 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:46:27.973534 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d5af57-b00f-4066-9b06-c10ffb56a86c None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:28.006207 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bff16c-3bce-4a51-9f6c-380ae2f21a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:28.006862 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bff16c-3bce-4a51-9f6c-380ae2f21a32 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:28.007141 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bff16c-3bce-4a51-9f6c-380ae2f21a32 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:28.007413 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bff16c-3bce-4a51-9f6c-380ae2f21a32 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:28.008177 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bff16c-3bce-4a51-9f6c-380ae2f21a32 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:46:28.008478 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bff16c-3bce-4a51-9f6c-380ae2f21a32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:28.009834 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bff16c-3bce-4a51-9f6c-380ae2f21a32 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:28.017548 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 568/1142] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Jun 10 12:46:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:46:28.023229 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3391826c-c92c-4ace-a8d1-0aa53429f645 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:28.118873 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3391826c-c92c-4ace-a8d1-0aa53429f645 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:28.151403 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e807ed3-6a66-4d88-a588-5b938f03c857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:28.152275 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e807ed3-6a66-4d88-a588-5b938f03c857 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:28.152559 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e807ed3-6a66-4d88-a588-5b938f03c857 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:28.152833 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e807ed3-6a66-4d88-a588-5b938f03c857 None admin] PATH_INFO: `/v3/users/be30dabfc6cd4221b7f667ace2b1c622` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:28.166213 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e807ed3-6a66-4d88-a588-5b938f03c857 None admin] RBAC: Authorizing `identity:get_user(user_id=be30dabfc6cd4221b7f667ace2b1c622)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:28.167660 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e807ed3-6a66-4d88-a588-5b938f03c857 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:28.169321 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 575/1143] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Jun 10 12:46:28 2022] GET /identity/v3/users/be30dabfc6cd4221b7f667ace2b1c622 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:46:28.175017 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bff16c-3bce-4a51-9f6c-380ae2f21a32 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:28.273024 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bff16c-3bce-4a51-9f6c-380ae2f21a32 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:28.300678 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:46:28.306064 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dd29a2-1389-4a9d-ba80-9987d95e201d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:28.306702 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dd29a2-1389-4a9d-ba80-9987d95e201d None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:28.306989 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dd29a2-1389-4a9d-ba80-9987d95e201d None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:28.307268 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dd29a2-1389-4a9d-ba80-9987d95e201d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:28.372075 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dd29a2-1389-4a9d-ba80-9987d95e201d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:28.373455 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dd29a2-1389-4a9d-ba80-9987d95e201d None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:28.468116 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 569/1144] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Jun 10 12:46:28 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 10 12:46:28.474569 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e807ed3-6a66-4d88-a588-5b938f03c857 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:28.500993 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:46:28.569736 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e807ed3-6a66-4d88-a588-5b938f03c857 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:28.602296 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0a7906-fb3b-4199-bdcf-a6f0b0558cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:28.603137 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0a7906-fb3b-4199-bdcf-a6f0b0558cab None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:28.603421 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0a7906-fb3b-4199-bdcf-a6f0b0558cab None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:28.603721 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0a7906-fb3b-4199-bdcf-a6f0b0558cab None admin] PATH_INFO: `/v3/projects/372b285d14354a2ca89f59a804f5f1e8` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:28.604920 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0a7906-fb3b-4199-bdcf-a6f0b0558cab None admin] RBAC: Authorizing `identity:get_project(project_id=372b285d14354a2ca89f59a804f5f1e8)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:28.606339 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0a7906-fb3b-4199-bdcf-a6f0b0558cab None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:28.608049 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 576/1145] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Jun 10 12:46:28 2022] GET /identity/v3/projects/372b285d14354a2ca89f59a804f5f1e8 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:46:28.613933 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dd29a2-1389-4a9d-ba80-9987d95e201d None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:28.642853 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:46:28.709744 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dd29a2-1389-4a9d-ba80-9987d95e201d None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:28.742224 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cd3ba8-9d6f-42a4-8032-5814477dfdf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:28.742861 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cd3ba8-9d6f-42a4-8032-5814477dfdf7 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:28.743141 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cd3ba8-9d6f-42a4-8032-5814477dfdf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:28.743414 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cd3ba8-9d6f-42a4-8032-5814477dfdf7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:28.744279 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cd3ba8-9d6f-42a4-8032-5814477dfdf7 None admin] RBAC: Adding query filter params (role.id=b43d5a8247e244869c2c70f986f42e8a, scope.project.id=372b285d14354a2ca89f59a804f5f1e8, user.id=be30dabfc6cd4221b7f667ace2b1c622) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:46:28.744582 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cd3ba8-9d6f-42a4-8032-5814477dfdf7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:28.745998 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cd3ba8-9d6f-42a4-8032-5814477dfdf7 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:28.751228 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 570/1146] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Jun 10 12:46:28 2022] GET /identity/v3/role_assignments?user.id=be30dabfc6cd4221b7f667ace2b1c622&scope.project.id=372b285d14354a2ca89f59a804f5f1e8&role.id=b43d5a8247e244869c2c70f986f42e8a => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:46:30.834174 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fde383d-e820-43a6-b953-64eaf89da3ad None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:30.834716 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fde383d-e820-43a6-b953-64eaf89da3ad None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:30.834716 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fde383d-e820-43a6-b953-64eaf89da3ad None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:30.836135 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 577/1147] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:46:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:46:30.841532 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ee4775-02a6-4799-954e-054b9932d637 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:30.841938 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ee4775-02a6-4799-954e-054b9932d637 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:30.842079 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ee4775-02a6-4799-954e-054b9932d637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:30.858797 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-79ee4775-02a6-4799-954e-054b9932d637 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:46:30.859993 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-79ee4775-02a6-4799-954e-054b9932d637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:46:30.923681 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 571/1148] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:46:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 10 12:46:30.946809 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fde383d-e820-43a6-b953-64eaf89da3ad None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:31.038347 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:46:31.044875 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fde383d-e820-43a6-b953-64eaf89da3ad None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:31.078611 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f36b540-cf49-4ac6-a947-d5987b418f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:31.079393 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f36b540-cf49-4ac6-a947-d5987b418f9c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:31.079679 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f36b540-cf49-4ac6-a947-d5987b418f9c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:31.079985 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f36b540-cf49-4ac6-a947-d5987b418f9c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:31.090307 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f36b540-cf49-4ac6-a947-d5987b418f9c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:46:31.098365 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-2f36b540-cf49-4ac6-a947-d5987b418f9c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 10 12:46:31.098365 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:46:31.098365 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:46:31.098365 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:46:31.098365 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:46:31.098365 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:46:31.098365 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:46:31.098365 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:46:31.098365 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:46:31.098365 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:46:31.098365 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:46:31.098365 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:46:31.098365 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:46:31.098365 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:46:31.098365 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:46:31.098365 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:46:31.099455 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:46:31.099455 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:46:31.099455 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:46:31.099455 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:46:31.099455 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:46:31.099455 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:46:31.099455 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:46:31.099455 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:46:31.099455 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:46:31.099455 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:46:31.099455 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:46:31.099455 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:46:31.099455 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:46:31.099455 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:46:31.099455 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:46:31.099455 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:46:31.100553 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:46:31.100553 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:46:31.100553 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:46:31.100553 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:46:31.100553 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:46:31.100553 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:46:31.100553 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:46:31.100553 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:46:31.100553 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:46:31.100553 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:46:31.100553 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 10 12:46:31.100553 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:46:31.100553 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 578/1149] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Jun 10 12:46:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:46:31.105495 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ee4775-02a6-4799-954e-054b9932d637 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:31.128854 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:46:31.198863 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ee4775-02a6-4799-954e-054b9932d637 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:31.231318 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b92fb3-ead6-4de8-8fbc-e0a0c868177e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:31.232000 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b92fb3-ead6-4de8-8fbc-e0a0c868177e None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:31.232282 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b92fb3-ead6-4de8-8fbc-e0a0c868177e None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:31.232552 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b92fb3-ead6-4de8-8fbc-e0a0c868177e None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:31.233291 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b92fb3-ead6-4de8-8fbc-e0a0c868177e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:46:31.233592 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b92fb3-ead6-4de8-8fbc-e0a0c868177e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:31.234942 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b92fb3-ead6-4de8-8fbc-e0a0c868177e None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:31.242662 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 572/1150] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Jun 10 12:46:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:46:31.248291 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f36b540-cf49-4ac6-a947-d5987b418f9c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:31.343227 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f36b540-cf49-4ac6-a947-d5987b418f9c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:31.375013 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e39c578-580b-4a3e-8ae4-231db2c564b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:31.375872 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e39c578-580b-4a3e-8ae4-231db2c564b1 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:31.376163 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e39c578-580b-4a3e-8ae4-231db2c564b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:31.376437 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e39c578-580b-4a3e-8ae4-231db2c564b1 None admin] PATH_INFO: `/v3/users/be30dabfc6cd4221b7f667ace2b1c622` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:31.377613 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e39c578-580b-4a3e-8ae4-231db2c564b1 None admin] RBAC: Authorizing `identity:get_user(user_id=be30dabfc6cd4221b7f667ace2b1c622)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:31.379015 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e39c578-580b-4a3e-8ae4-231db2c564b1 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:31.380618 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 579/1151] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Jun 10 12:46:31 2022] GET /identity/v3/users/be30dabfc6cd4221b7f667ace2b1c622 => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:46:31.386139 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b92fb3-ead6-4de8-8fbc-e0a0c868177e None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:31.468085 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:46:31.482696 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b92fb3-ead6-4de8-8fbc-e0a0c868177e None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:31.515169 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb47d740-f49f-49bf-9440-ec27b92187ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:31.516035 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb47d740-f49f-49bf-9440-ec27b92187ff None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:31.516319 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb47d740-f49f-49bf-9440-ec27b92187ff None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:31.516597 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb47d740-f49f-49bf-9440-ec27b92187ff None admin] PATH_INFO: `/v3/projects/372b285d14354a2ca89f59a804f5f1e8` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:31.517764 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb47d740-f49f-49bf-9440-ec27b92187ff None admin] RBAC: Authorizing `identity:get_project(project_id=372b285d14354a2ca89f59a804f5f1e8)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:31.519711 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb47d740-f49f-49bf-9440-ec27b92187ff None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:31.521403 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 573/1152] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Jun 10 12:46:31 2022] GET /identity/v3/projects/372b285d14354a2ca89f59a804f5f1e8 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:46:31.527070 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e39c578-580b-4a3e-8ae4-231db2c564b1 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:31.551882 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:46:31.622719 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e39c578-580b-4a3e-8ae4-231db2c564b1 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:31.655062 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594dafed-5a5f-4b46-acf7-16515d6ee33d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:31.655955 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594dafed-5a5f-4b46-acf7-16515d6ee33d None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:31.656243 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594dafed-5a5f-4b46-acf7-16515d6ee33d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:31.656523 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594dafed-5a5f-4b46-acf7-16515d6ee33d None admin] PATH_INFO: `/v3/projects/372b285d14354a2ca89f59a804f5f1e8/users/be30dabfc6cd4221b7f667ace2b1c622/roles/b43d5a8247e244869c2c70f986f42e8a` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:31.658438 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594dafed-5a5f-4b46-acf7-16515d6ee33d None admin] RBAC: Authorizing `identity:create_grant(project_id=372b285d14354a2ca89f59a804f5f1e8, user_id=be30dabfc6cd4221b7f667ace2b1c622, role_id=b43d5a8247e244869c2c70f986f42e8a)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:31.660155 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594dafed-5a5f-4b46-acf7-16515d6ee33d None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:31.758361 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 580/1153] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Jun 10 12:46:31 2022] PUT /identity/v3/projects/372b285d14354a2ca89f59a804f5f1e8/users/be30dabfc6cd4221b7f667ace2b1c622/roles/b43d5a8247e244869c2c70f986f42e8a => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 12:46:33.848391 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4166a7f-6b36-44c0-b8c4-d1ac9769fc2f None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:33.848947 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4166a7f-6b36-44c0-b8c4-d1ac9769fc2f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:33.848947 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4166a7f-6b36-44c0-b8c4-d1ac9769fc2f None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:33.850312 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 574/1154] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:46:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 10 12:46:33.855596 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542a3eaa-53aa-4b91-9577-394c88d04b75 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:33.856026 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542a3eaa-53aa-4b91-9577-394c88d04b75 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:33.856178 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542a3eaa-53aa-4b91-9577-394c88d04b75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:33.873705 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-542a3eaa-53aa-4b91-9577-394c88d04b75 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:46:33.874874 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-542a3eaa-53aa-4b91-9577-394c88d04b75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:46:33.938271 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 581/1155] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:46:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 10 12:46:33.961137 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4166a7f-6b36-44c0-b8c4-d1ac9769fc2f None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:34.059831 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4166a7f-6b36-44c0-b8c4-d1ac9769fc2f None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:34.092598 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4422845-37bf-4705-95f4-ea3c3bbaafe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:34.093380 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4422845-37bf-4705-95f4-ea3c3bbaafe4 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:34.093661 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4422845-37bf-4705-95f4-ea3c3bbaafe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:34.093938 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4422845-37bf-4705-95f4-ea3c3bbaafe4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:34.104352 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4422845-37bf-4705-95f4-ea3c3bbaafe4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:46:34.111823 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-f4422845-37bf-4705-95f4-ea3c3bbaafe4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 10 12:46:34.111823 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:46:34.111823 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:46:34.111823 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:46:34.111823 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:46:34.111823 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:46:34.111823 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:46:34.111823 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:46:34.111823 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:46:34.111823 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:46:34.111823 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:46:34.111823 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:46:34.111823 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:46:34.111823 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:46:34.111823 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:46:34.111823 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:46:34.113053 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:46:34.113053 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:46:34.113053 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:46:34.113053 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:46:34.113053 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:46:34.113053 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:46:34.113053 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:46:34.113053 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:46:34.113053 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:46:34.113053 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:46:34.113053 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:46:34.113053 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:46:34.113053 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:46:34.113053 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:46:34.113053 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:46:34.113053 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:46:34.114124 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:46:34.114124 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:46:34.114124 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:46:34.114124 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:46:34.114124 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:46:34.114124 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:46:34.114124 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:46:34.114124 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:46:34.114124 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:46:34.114124 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:46:34.114124 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 10 12:46:34.114124 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:46:34.114124 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 575/1156] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Jun 10 12:46:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:46:34.119089 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542a3eaa-53aa-4b91-9577-394c88d04b75 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:34.133750 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:46:34.215555 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542a3eaa-53aa-4b91-9577-394c88d04b75 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:34.248085 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dfb882-586c-4f14-bb31-cd3dc73c4e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:34.248735 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dfb882-586c-4f14-bb31-cd3dc73c4e62 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:34.249015 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dfb882-586c-4f14-bb31-cd3dc73c4e62 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:34.249286 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dfb882-586c-4f14-bb31-cd3dc73c4e62 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:34.250018 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dfb882-586c-4f14-bb31-cd3dc73c4e62 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:46:34.250319 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dfb882-586c-4f14-bb31-cd3dc73c4e62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:34.252040 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dfb882-586c-4f14-bb31-cd3dc73c4e62 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:34.259770 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 582/1157] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Jun 10 12:46:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:46:34.265419 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4422845-37bf-4705-95f4-ea3c3bbaafe4 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:34.281649 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:46:34.368672 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4422845-37bf-4705-95f4-ea3c3bbaafe4 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:34.401794 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acce164b-4caa-4d1f-b61d-fa3c36ec2872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:34.402648 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acce164b-4caa-4d1f-b61d-fa3c36ec2872 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:34.402932 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acce164b-4caa-4d1f-b61d-fa3c36ec2872 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:34.403205 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acce164b-4caa-4d1f-b61d-fa3c36ec2872 None admin] PATH_INFO: `/v3/users/be30dabfc6cd4221b7f667ace2b1c622` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:34.404399 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acce164b-4caa-4d1f-b61d-fa3c36ec2872 None admin] RBAC: Authorizing `identity:get_user(user_id=be30dabfc6cd4221b7f667ace2b1c622)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:34.405831 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acce164b-4caa-4d1f-b61d-fa3c36ec2872 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:34.407426 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 576/1158] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Jun 10 12:46:34 2022] GET /identity/v3/users/be30dabfc6cd4221b7f667ace2b1c622 => generated 286 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:46:34.413997 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dfb882-586c-4f14-bb31-cd3dc73c4e62 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:34.486807 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:46:34.509707 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dfb882-586c-4f14-bb31-cd3dc73c4e62 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:34.541682 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eefc9e-206d-4ea4-97c7-b9c3ad080ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:34.542311 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eefc9e-206d-4ea4-97c7-b9c3ad080ff1 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:34.542594 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eefc9e-206d-4ea4-97c7-b9c3ad080ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:34.542869 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eefc9e-206d-4ea4-97c7-b9c3ad080ff1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:34.605968 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1eefc9e-206d-4ea4-97c7-b9c3ad080ff1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:34.607377 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1eefc9e-206d-4ea4-97c7-b9c3ad080ff1 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:34.678828 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:46:34.700895 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 583/1159] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Jun 10 12:46:34 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 10 12:46:34.707371 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acce164b-4caa-4d1f-b61d-fa3c36ec2872 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:34.785578 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:46:34.802628 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acce164b-4caa-4d1f-b61d-fa3c36ec2872 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:34.835622 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e43b9e-c28a-41c1-bcee-f42f92c97a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:34.836483 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e43b9e-c28a-41c1-bcee-f42f92c97a9b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:34.836768 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e43b9e-c28a-41c1-bcee-f42f92c97a9b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:34.837043 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e43b9e-c28a-41c1-bcee-f42f92c97a9b None admin] PATH_INFO: `/v3/projects/372b285d14354a2ca89f59a804f5f1e8` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:34.838201 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e43b9e-c28a-41c1-bcee-f42f92c97a9b None admin] RBAC: Authorizing `identity:get_project(project_id=372b285d14354a2ca89f59a804f5f1e8)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:34.839625 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e43b9e-c28a-41c1-bcee-f42f92c97a9b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:34.841303 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 577/1160] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Jun 10 12:46:34 2022] GET /identity/v3/projects/372b285d14354a2ca89f59a804f5f1e8 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:46:34.847312 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eefc9e-206d-4ea4-97c7-b9c3ad080ff1 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:34.943511 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eefc9e-206d-4ea4-97c7-b9c3ad080ff1 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:34.975472 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ae7613-e694-4d02-baa6-e62b5643af81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:34.976139 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ae7613-e694-4d02-baa6-e62b5643af81 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:34.976425 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ae7613-e694-4d02-baa6-e62b5643af81 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:34.976702 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ae7613-e694-4d02-baa6-e62b5643af81 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:34.977523 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ae7613-e694-4d02-baa6-e62b5643af81 None admin] RBAC: Adding query filter params (role.id=b43d5a8247e244869c2c70f986f42e8a, scope.project.id=372b285d14354a2ca89f59a804f5f1e8, user.id=be30dabfc6cd4221b7f667ace2b1c622) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:46:34.977831 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ae7613-e694-4d02-baa6-e62b5643af81 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:34.979238 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ae7613-e694-4d02-baa6-e62b5643af81 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:34.984855 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 584/1161] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Jun 10 12:46:34 2022] GET /identity/v3/role_assignments?user.id=be30dabfc6cd4221b7f667ace2b1c622&scope.project.id=372b285d14354a2ca89f59a804f5f1e8&role.id=b43d5a8247e244869c2c70f986f42e8a => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:46:37.079915 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6e10ac-8968-4335-aee5-1293f8324952 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:37.080956 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6e10ac-8968-4335-aee5-1293f8324952 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:37.081095 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6e10ac-8968-4335-aee5-1293f8324952 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:37.082376 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 578/1162] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:46:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:46:37.087362 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec701b8-8cfb-46e3-a176-47fdda99c1ed None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:37.087788 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec701b8-8cfb-46e3-a176-47fdda99c1ed None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:37.087944 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec701b8-8cfb-46e3-a176-47fdda99c1ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:37.105194 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-7ec701b8-8cfb-46e3-a176-47fdda99c1ed None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:46:37.106372 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-7ec701b8-8cfb-46e3-a176-47fdda99c1ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:46:37.138894 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:46:37.171143 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 585/1163] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:46:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 10 12:46:37.193350 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6e10ac-8968-4335-aee5-1293f8324952 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:37.292716 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6e10ac-8968-4335-aee5-1293f8324952 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:37.324980 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e66e62-23af-408d-b981-7bffa1652565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:37.325613 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e66e62-23af-408d-b981-7bffa1652565 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:37.325897 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e66e62-23af-408d-b981-7bffa1652565 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:37.326175 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e66e62-23af-408d-b981-7bffa1652565 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:37.344233 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:46:37.391796 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e66e62-23af-408d-b981-7bffa1652565 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:37.393745 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e66e62-23af-408d-b981-7bffa1652565 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:37.487962 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 579/1164] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Jun 10 12:46:37 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 10 12:46:37.493690 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec701b8-8cfb-46e3-a176-47fdda99c1ed None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:37.588286 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec701b8-8cfb-46e3-a176-47fdda99c1ed None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:37.620073 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68af7721-24e3-4401-9ce4-6a076806e573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:37.620854 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af7721-24e3-4401-9ce4-6a076806e573 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:37.621138 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af7721-24e3-4401-9ce4-6a076806e573 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:37.621412 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af7721-24e3-4401-9ce4-6a076806e573 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:37.628364 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68af7721-24e3-4401-9ce4-6a076806e573 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:37.629779 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68af7721-24e3-4401-9ce4-6a076806e573 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:37.644146 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-68af7721-24e3-4401-9ce4-6a076806e573 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 10 12:46:37.644146 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:46:37.644146 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 10 12:46:37.644146 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 10 12:46:37.644146 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 10 12:46:37.644146 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 10 12:46:37.644146 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 10 12:46:37.644146 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 10 12:46:37.644146 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 10 12:46:37.644146 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:46:37.644146 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 10 12:46:37.644146 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:46:37.644146 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:46:37.644146 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:46:37.644146 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:46:37.644146 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:46:37.644146 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:46:37.645338 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:46:37.645338 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:46:37.645338 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:46:37.645338 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:46:37.645338 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:46:37.645338 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:46:37.645338 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 10 12:46:37.645338 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 10 12:46:37.645338 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 10 12:46:37.645338 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 10 12:46:37.645338 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:46:37.645338 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:46:37.645338 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:46:37.645338 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:46:37.645338 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:46:37.645338 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:46:37.646389 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:46:37.646389 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:46:37.646389 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:46:37.646389 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:46:37.646389 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:46:37.646389 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:46:37.646389 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:46:37.646389 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:46:37.646389 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:46:37.646389 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:46:37.646389 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 10 12:46:37.646389 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 10 12:46:37.646389 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 10 12:46:37.646389 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:46:37.646389 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 586/1165] 10.222.0.27 () {56 vars in 1117 bytes} [Fri Jun 10 12:46:37 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:46:37.651078 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e66e62-23af-408d-b981-7bffa1652565 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:37.699967 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:46:37.748104 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e66e62-23af-408d-b981-7bffa1652565 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:37.781002 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5640c3d5-10ef-4886-b8ec-530168c6ce25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:37.781657 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5640c3d5-10ef-4886-b8ec-530168c6ce25 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:37.781943 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5640c3d5-10ef-4886-b8ec-530168c6ce25 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:37.782219 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5640c3d5-10ef-4886-b8ec-530168c6ce25 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:37.782941 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5640c3d5-10ef-4886-b8ec-530168c6ce25 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:46:37.783247 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5640c3d5-10ef-4886-b8ec-530168c6ce25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:37.784935 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5640c3d5-10ef-4886-b8ec-530168c6ce25 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:37.791835 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 580/1166] 10.222.0.27 () {56 vars in 1115 bytes} [Fri Jun 10 12:46:37 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:46:39.851761 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282dd4bd-258a-4ba5-b30f-65dda7119d8b None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:39.852333 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282dd4bd-258a-4ba5-b30f-65dda7119d8b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:39.852333 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282dd4bd-258a-4ba5-b30f-65dda7119d8b None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:39.853704 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 587/1167] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:46:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:46:39.858780 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e6bd49-71a2-4b82-8456-bfcde9c83d60 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:39.859181 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e6bd49-71a2-4b82-8456-bfcde9c83d60 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:39.859354 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e6bd49-71a2-4b82-8456-bfcde9c83d60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:39.876018 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-e9e6bd49-71a2-4b82-8456-bfcde9c83d60 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:46:39.877745 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e9e6bd49-71a2-4b82-8456-bfcde9c83d60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:46:39.941805 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 581/1168] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:46:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 10 12:46:39.964367 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282dd4bd-258a-4ba5-b30f-65dda7119d8b None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:39.993618 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:46:40.060563 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282dd4bd-258a-4ba5-b30f-65dda7119d8b None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:40.092634 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc132cfc-ab40-4f5e-9f9b-40120bdc1a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:40.093269 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc132cfc-ab40-4f5e-9f9b-40120bdc1a3c None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:40.093548 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc132cfc-ab40-4f5e-9f9b-40120bdc1a3c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:40.093821 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc132cfc-ab40-4f5e-9f9b-40120bdc1a3c None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:40.094547 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc132cfc-ab40-4f5e-9f9b-40120bdc1a3c None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:40.095957 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc132cfc-ab40-4f5e-9f9b-40120bdc1a3c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:40.198648 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 588/1169] 10.222.0.27 () {60 vars in 1138 bytes} [Fri Jun 10 12:46:39 2022] POST /identity/v3/domains => generated 267 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 12:46:42.321078 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43993b0-266f-4283-93f1-e1e348e8e1b3 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:42.321666 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43993b0-266f-4283-93f1-e1e348e8e1b3 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:42.321666 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43993b0-266f-4283-93f1-e1e348e8e1b3 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:42.322925 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 582/1170] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:46:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:46:42.328168 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8102b77f-e85f-4ec1-9a61-120f530e2ca3 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:42.328598 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8102b77f-e85f-4ec1-9a61-120f530e2ca3 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:42.328734 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8102b77f-e85f-4ec1-9a61-120f530e2ca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:42.345573 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-8102b77f-e85f-4ec1-9a61-120f530e2ca3 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:46:42.346775 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8102b77f-e85f-4ec1-9a61-120f530e2ca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:46:42.410214 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 589/1171] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:46:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 10 12:46:42.433595 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43993b0-266f-4283-93f1-e1e348e8e1b3 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:42.479634 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:46:42.531608 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43993b0-266f-4283-93f1-e1e348e8e1b3 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:42.565828 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621bf43e-1bb0-4f22-ad99-77c130880638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:42.566671 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621bf43e-1bb0-4f22-ad99-77c130880638 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:42.566954 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621bf43e-1bb0-4f22-ad99-77c130880638 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:42.567232 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621bf43e-1bb0-4f22-ad99-77c130880638 None admin] PATH_INFO: `/v3/domains/8491172f84604dfbb74a828c69246355` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:42.577415 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621bf43e-1bb0-4f22-ad99-77c130880638 None admin] RBAC: Authorizing `identity:get_domain(domain_id=8491172f84604dfbb74a828c69246355)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:42.578815 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621bf43e-1bb0-4f22-ad99-77c130880638 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:42.580466 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 583/1172] 10.222.0.27 () {56 vars in 1183 bytes} [Fri Jun 10 12:46:42 2022] GET /identity/v3/domains/8491172f84604dfbb74a828c69246355 => generated 267 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:46:42.586295 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8102b77f-e85f-4ec1-9a61-120f530e2ca3 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:42.606421 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:46:42.681090 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8102b77f-e85f-4ec1-9a61-120f530e2ca3 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:42.713256 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac45a9f-b268-4ef6-800e-9de0027c9b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:42.713904 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac45a9f-b268-4ef6-800e-9de0027c9b2b None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:42.714190 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac45a9f-b268-4ef6-800e-9de0027c9b2b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:42.714471 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac45a9f-b268-4ef6-800e-9de0027c9b2b None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:42.715174 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac45a9f-b268-4ef6-800e-9de0027c9b2b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:42.716610 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac45a9f-b268-4ef6-800e-9de0027c9b2b None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:42.804560 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 590/1173] 10.222.0.27 () {60 vars in 1141 bytes} [Fri Jun 10 12:46:42 2022] POST /identity/v3/projects => generated 361 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 10 12:46:44.918886 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f5d399-477f-409c-8a53-5dd9036e85b9 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:44.919415 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f5d399-477f-409c-8a53-5dd9036e85b9 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:44.919415 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f5d399-477f-409c-8a53-5dd9036e85b9 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:44.920860 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 584/1174] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:46:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:46:44.926570 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbe306c-2f68-47fb-a89a-3108479397a9 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:44.927007 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbe306c-2f68-47fb-a89a-3108479397a9 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:44.927136 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbe306c-2f68-47fb-a89a-3108479397a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:44.944005 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-8bbe306c-2f68-47fb-a89a-3108479397a9 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:46:44.945197 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8bbe306c-2f68-47fb-a89a-3108479397a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:46:45.008490 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 591/1175] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:46:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 10 12:46:45.032447 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f5d399-477f-409c-8a53-5dd9036e85b9 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:45.133823 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f5d399-477f-409c-8a53-5dd9036e85b9 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:45.136035 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:46:45.166511 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052790ba-8b21-4976-8919-155595b9110e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:45.167359 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052790ba-8b21-4976-8919-155595b9110e None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:45.167643 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052790ba-8b21-4976-8919-155595b9110e None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:45.167955 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052790ba-8b21-4976-8919-155595b9110e None admin] PATH_INFO: `/v3/domains/8491172f84604dfbb74a828c69246355` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:45.169122 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052790ba-8b21-4976-8919-155595b9110e None admin] RBAC: Authorizing `identity:get_domain(domain_id=8491172f84604dfbb74a828c69246355)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:45.170538 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052790ba-8b21-4976-8919-155595b9110e None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:45.172163 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 585/1176] 10.222.0.27 () {56 vars in 1183 bytes} [Fri Jun 10 12:46:45 2022] GET /identity/v3/domains/8491172f84604dfbb74a828c69246355 => generated 267 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:46:45.178356 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbe306c-2f68-47fb-a89a-3108479397a9 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:45.180527 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:46:45.273111 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbe306c-2f68-47fb-a89a-3108479397a9 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:45.306906 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94155b7a-ba9c-42e9-a8b3-837c60ef7870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:45.307559 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94155b7a-ba9c-42e9-a8b3-837c60ef7870 None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:45.307844 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94155b7a-ba9c-42e9-a8b3-837c60ef7870 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:45.308154 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94155b7a-ba9c-42e9-a8b3-837c60ef7870 None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:45.308855 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94155b7a-ba9c-42e9-a8b3-837c60ef7870 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:45.310261 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94155b7a-ba9c-42e9-a8b3-837c60ef7870 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:45.397422 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 592/1177] 10.222.0.27 () {60 vars in 1132 bytes} [Fri Jun 10 12:46:45 2022] POST /identity/v3/users => generated 311 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 12:46:47.515303 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dd8592-a081-4654-a630-2bfe2479b48b None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:47.515954 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dd8592-a081-4654-a630-2bfe2479b48b None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:47.515954 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dd8592-a081-4654-a630-2bfe2479b48b None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:47.518056 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 586/1178] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:46:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:46:47.523256 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c868229a-aa1f-4d69-a0e6-fca6f3f2bd4a None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:47.523674 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c868229a-aa1f-4d69-a0e6-fca6f3f2bd4a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:47.523816 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c868229a-aa1f-4d69-a0e6-fca6f3f2bd4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:47.540277 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-c868229a-aa1f-4d69-a0e6-fca6f3f2bd4a None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:46:47.541451 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c868229a-aa1f-4d69-a0e6-fca6f3f2bd4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:46:47.592131 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:46:47.604983 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 593/1179] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:46:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 10 12:46:47.629950 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dd8592-a081-4654-a630-2bfe2479b48b None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:47.727173 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dd8592-a081-4654-a630-2bfe2479b48b None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:47.759785 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0666dab5-ddbd-478d-a9cb-b3df3b21e893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:47.760584 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0666dab5-ddbd-478d-a9cb-b3df3b21e893 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:47.760864 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0666dab5-ddbd-478d-a9cb-b3df3b21e893 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:47.761134 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0666dab5-ddbd-478d-a9cb-b3df3b21e893 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:47.770787 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0666dab5-ddbd-478d-a9cb-b3df3b21e893 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:46:47.778182 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-0666dab5-ddbd-478d-a9cb-b3df3b21e893 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 10 12:46:47.778182 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:46:47.778182 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:46:47.778182 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:46:47.778182 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:46:47.778182 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:46:47.778182 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:46:47.778182 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:46:47.778182 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:46:47.778182 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:46:47.778182 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:46:47.778182 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:46:47.778182 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:46:47.778182 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:46:47.778182 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:46:47.778182 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:46:47.779378 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:46:47.779378 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:46:47.779378 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:46:47.779378 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:46:47.779378 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:46:47.779378 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:46:47.779378 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:46:47.779378 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:46:47.779378 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:46:47.779378 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:46:47.779378 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:46:47.779378 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:46:47.779378 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:46:47.779378 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:46:47.779378 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:46:47.779378 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:46:47.780388 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:46:47.780388 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:46:47.780388 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:46:47.780388 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:46:47.780388 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:46:47.780388 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:46:47.780388 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:46:47.780388 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:46:47.780388 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:46:47.780388 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:46:47.780388 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 10 12:46:47.780388 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:46:47.780388 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 587/1180] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Jun 10 12:46:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:46:47.785478 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c868229a-aa1f-4d69-a0e6-fca6f3f2bd4a None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:47.880391 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c868229a-aa1f-4d69-a0e6-fca6f3f2bd4a None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:47.912687 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72f2d72-f87b-46f0-8734-99a8f79949f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:47.913357 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72f2d72-f87b-46f0-8734-99a8f79949f6 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:47.913639 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72f2d72-f87b-46f0-8734-99a8f79949f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:47.913915 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72f2d72-f87b-46f0-8734-99a8f79949f6 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:47.914657 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72f2d72-f87b-46f0-8734-99a8f79949f6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:46:47.914962 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72f2d72-f87b-46f0-8734-99a8f79949f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:47.916357 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72f2d72-f87b-46f0-8734-99a8f79949f6 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:47.924927 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 594/1181] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Jun 10 12:46:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:46:47.930579 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0666dab5-ddbd-478d-a9cb-b3df3b21e893 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:47.961387 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:46:48.026967 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0666dab5-ddbd-478d-a9cb-b3df3b21e893 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:48.060065 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a9d484-f40f-4b9b-9c36-75e69a8bfe0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:48.060904 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a9d484-f40f-4b9b-9c36-75e69a8bfe0b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:48.061181 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a9d484-f40f-4b9b-9c36-75e69a8bfe0b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:48.061451 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a9d484-f40f-4b9b-9c36-75e69a8bfe0b None admin] PATH_INFO: `/v3/users/dbfccbed92dd456cbcdc7c73fac58bb7` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:48.073883 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a9d484-f40f-4b9b-9c36-75e69a8bfe0b None admin] RBAC: Authorizing `identity:get_user(user_id=dbfccbed92dd456cbcdc7c73fac58bb7)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:48.075332 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a9d484-f40f-4b9b-9c36-75e69a8bfe0b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:48.076958 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 588/1182] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Jun 10 12:46:47 2022] GET /identity/v3/users/dbfccbed92dd456cbcdc7c73fac58bb7 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:46:48.082538 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72f2d72-f87b-46f0-8734-99a8f79949f6 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:48.131410 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:46:48.180307 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72f2d72-f87b-46f0-8734-99a8f79949f6 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:48.211991 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b37b71c-a917-426b-bcbd-8a4c7f7dc9c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:48.212625 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b37b71c-a917-426b-bcbd-8a4c7f7dc9c0 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:48.212907 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b37b71c-a917-426b-bcbd-8a4c7f7dc9c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:48.213209 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b37b71c-a917-426b-bcbd-8a4c7f7dc9c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:48.277596 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b37b71c-a917-426b-bcbd-8a4c7f7dc9c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:48.278956 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b37b71c-a917-426b-bcbd-8a4c7f7dc9c0 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:48.325638 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:46:48.373924 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 595/1183] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Jun 10 12:46:48 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 10 12:46:48.379880 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a9d484-f40f-4b9b-9c36-75e69a8bfe0b None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:48.467185 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:46:48.477050 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a9d484-f40f-4b9b-9c36-75e69a8bfe0b None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:48.510122 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6b24fe-8c2e-4e94-8c3b-2448f1616919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:48.510959 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6b24fe-8c2e-4e94-8c3b-2448f1616919 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:48.511246 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6b24fe-8c2e-4e94-8c3b-2448f1616919 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:48.511524 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6b24fe-8c2e-4e94-8c3b-2448f1616919 None admin] PATH_INFO: `/v3/projects/3ed6c933db4941b096e7cb7c9f37415e` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:48.512738 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6b24fe-8c2e-4e94-8c3b-2448f1616919 None admin] RBAC: Authorizing `identity:get_project(project_id=3ed6c933db4941b096e7cb7c9f37415e)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:48.514162 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6b24fe-8c2e-4e94-8c3b-2448f1616919 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:48.515829 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 589/1184] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Jun 10 12:46:48 2022] GET /identity/v3/projects/3ed6c933db4941b096e7cb7c9f37415e => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:46:48.522246 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b37b71c-a917-426b-bcbd-8a4c7f7dc9c0 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:48.615489 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b37b71c-a917-426b-bcbd-8a4c7f7dc9c0 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:48.647481 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65497ef5-baaa-46df-9e45-bf60be45ae31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:48.648135 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65497ef5-baaa-46df-9e45-bf60be45ae31 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:48.648417 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65497ef5-baaa-46df-9e45-bf60be45ae31 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:48.648692 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65497ef5-baaa-46df-9e45-bf60be45ae31 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:48.649510 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65497ef5-baaa-46df-9e45-bf60be45ae31 None admin] RBAC: Adding query filter params (role.id=b43d5a8247e244869c2c70f986f42e8a, scope.project.id=3ed6c933db4941b096e7cb7c9f37415e, user.id=dbfccbed92dd456cbcdc7c73fac58bb7) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:46:48.649818 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65497ef5-baaa-46df-9e45-bf60be45ae31 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:48.651225 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65497ef5-baaa-46df-9e45-bf60be45ae31 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:48.656420 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 596/1185] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Jun 10 12:46:48 2022] GET /identity/v3/role_assignments?user.id=dbfccbed92dd456cbcdc7c73fac58bb7&scope.project.id=3ed6c933db4941b096e7cb7c9f37415e&role.id=b43d5a8247e244869c2c70f986f42e8a => generated 259 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:46:50.708829 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9913b6da-9733-4959-8bf7-ed880b892a71 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:50.709388 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9913b6da-9733-4959-8bf7-ed880b892a71 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:50.709388 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9913b6da-9733-4959-8bf7-ed880b892a71 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:50.710730 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 590/1186] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:46:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:46:50.715782 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fe04da-1fe1-4f4d-ad80-b07246eb8f67 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:50.716185 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fe04da-1fe1-4f4d-ad80-b07246eb8f67 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:50.716378 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fe04da-1fe1-4f4d-ad80-b07246eb8f67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:50.733166 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-a1fe04da-1fe1-4f4d-ad80-b07246eb8f67 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:46:50.734326 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a1fe04da-1fe1-4f4d-ad80-b07246eb8f67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:46:50.743734 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:46:50.797340 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 597/1187] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:46:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 10 12:46:50.819730 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9913b6da-9733-4959-8bf7-ed880b892a71 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:50.917225 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9913b6da-9733-4959-8bf7-ed880b892a71 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:50.950504 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30689d31-3708-4439-a4bf-0917a36b538e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:50.951280 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30689d31-3708-4439-a4bf-0917a36b538e None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:50.951564 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30689d31-3708-4439-a4bf-0917a36b538e None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:50.951838 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30689d31-3708-4439-a4bf-0917a36b538e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:50.961564 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30689d31-3708-4439-a4bf-0917a36b538e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:46:50.969019 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-30689d31-3708-4439-a4bf-0917a36b538e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 10 12:46:50.969019 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:46:50.969019 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:46:50.969019 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:46:50.969019 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:46:50.969019 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:46:50.969019 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:46:50.969019 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:46:50.969019 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:46:50.969019 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:46:50.969019 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:46:50.969019 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:46:50.969019 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:46:50.969019 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:46:50.969019 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:46:50.969019 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:46:50.970636 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:46:50.970636 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:46:50.970636 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:46:50.970636 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:46:50.970636 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:46:50.970636 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:46:50.970636 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:46:50.970636 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:46:50.970636 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:46:50.970636 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:46:50.970636 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:46:50.970636 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:46:50.970636 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:46:50.970636 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:46:50.970636 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:46:50.970636 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:46:50.971536 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:46:50.971536 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:46:50.971536 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:46:50.971536 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:46:50.971536 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:46:50.971536 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:46:50.971536 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:46:50.971536 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:46:50.971536 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:46:50.971536 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:46:50.971536 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 10 12:46:50.971536 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:46:50.971536 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 591/1188] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Jun 10 12:46:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:46:50.976808 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fe04da-1fe1-4f4d-ad80-b07246eb8f67 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:51.070935 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fe04da-1fe1-4f4d-ad80-b07246eb8f67 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:51.103004 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b0f921-73f6-4173-9dea-9551c3a3c7b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:51.103655 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b0f921-73f6-4173-9dea-9551c3a3c7b9 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:51.103985 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b0f921-73f6-4173-9dea-9551c3a3c7b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:51.104255 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b0f921-73f6-4173-9dea-9551c3a3c7b9 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:51.104992 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b0f921-73f6-4173-9dea-9551c3a3c7b9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:46:51.105298 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b0f921-73f6-4173-9dea-9551c3a3c7b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:51.106650 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b0f921-73f6-4173-9dea-9551c3a3c7b9 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:51.114956 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 598/1189] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Jun 10 12:46:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:46:51.120547 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30689d31-3708-4439-a4bf-0917a36b538e None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:51.139153 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:46:51.217987 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30689d31-3708-4439-a4bf-0917a36b538e None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:51.250115 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dc5f13-6d1c-45ec-8ddd-e070ba4aca86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:51.250973 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dc5f13-6d1c-45ec-8ddd-e070ba4aca86 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:51.251277 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dc5f13-6d1c-45ec-8ddd-e070ba4aca86 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:51.251552 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dc5f13-6d1c-45ec-8ddd-e070ba4aca86 None admin] PATH_INFO: `/v3/users/dbfccbed92dd456cbcdc7c73fac58bb7` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:51.252738 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dc5f13-6d1c-45ec-8ddd-e070ba4aca86 None admin] RBAC: Authorizing `identity:get_user(user_id=dbfccbed92dd456cbcdc7c73fac58bb7)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:51.254155 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dc5f13-6d1c-45ec-8ddd-e070ba4aca86 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:51.255734 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 592/1190] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Jun 10 12:46:51 2022] GET /identity/v3/users/dbfccbed92dd456cbcdc7c73fac58bb7 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:46:51.261603 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b0f921-73f6-4173-9dea-9551c3a3c7b9 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:51.270618 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:46:51.358526 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b0f921-73f6-4173-9dea-9551c3a3c7b9 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:51.390060 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd57273a-07c9-4412-a6a2-9ba08a33f682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:51.390902 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd57273a-07c9-4412-a6a2-9ba08a33f682 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:51.391185 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd57273a-07c9-4412-a6a2-9ba08a33f682 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:51.391462 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd57273a-07c9-4412-a6a2-9ba08a33f682 None admin] PATH_INFO: `/v3/projects/3ed6c933db4941b096e7cb7c9f37415e` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:51.393165 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd57273a-07c9-4412-a6a2-9ba08a33f682 None admin] RBAC: Authorizing `identity:get_project(project_id=3ed6c933db4941b096e7cb7c9f37415e)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:51.394572 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd57273a-07c9-4412-a6a2-9ba08a33f682 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:51.396265 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 599/1191] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Jun 10 12:46:51 2022] GET /identity/v3/projects/3ed6c933db4941b096e7cb7c9f37415e => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:46:51.401993 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dc5f13-6d1c-45ec-8ddd-e070ba4aca86 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:51.483286 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:46:51.496878 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dc5f13-6d1c-45ec-8ddd-e070ba4aca86 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:51.529512 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315bb170-8072-4696-b49f-c8b486c25b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:51.530403 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315bb170-8072-4696-b49f-c8b486c25b42 None admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:51.530685 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315bb170-8072-4696-b49f-c8b486c25b42 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:51.530958 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315bb170-8072-4696-b49f-c8b486c25b42 None admin] PATH_INFO: `/v3/projects/3ed6c933db4941b096e7cb7c9f37415e/users/dbfccbed92dd456cbcdc7c73fac58bb7/roles/b43d5a8247e244869c2c70f986f42e8a` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:51.533435 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315bb170-8072-4696-b49f-c8b486c25b42 None admin] RBAC: Authorizing `identity:create_grant(project_id=3ed6c933db4941b096e7cb7c9f37415e, user_id=dbfccbed92dd456cbcdc7c73fac58bb7, role_id=b43d5a8247e244869c2c70f986f42e8a)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:51.535084 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315bb170-8072-4696-b49f-c8b486c25b42 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:51.585768 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 593/1192] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Jun 10 12:46:51 2022] PUT /identity/v3/projects/3ed6c933db4941b096e7cb7c9f37415e/users/dbfccbed92dd456cbcdc7c73fac58bb7/roles/b43d5a8247e244869c2c70f986f42e8a => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 12:46:53.636214 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b34da3-1a0e-4077-b6be-3c26403560c2 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:53.636759 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b34da3-1a0e-4077-b6be-3c26403560c2 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:53.636759 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b34da3-1a0e-4077-b6be-3c26403560c2 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:53.638130 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 600/1193] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:46:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:46:53.643138 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9514343-b2bc-4b01-bdb3-29ec2e99c81d None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:53.643523 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9514343-b2bc-4b01-bdb3-29ec2e99c81d None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:53.643693 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9514343-b2bc-4b01-bdb3-29ec2e99c81d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:53.660082 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-f9514343-b2bc-4b01-bdb3-29ec2e99c81d None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:46:53.661245 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f9514343-b2bc-4b01-bdb3-29ec2e99c81d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:46:53.724329 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 594/1194] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:46:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 10 12:46:53.747117 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b34da3-1a0e-4077-b6be-3c26403560c2 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:53.820716 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:46:53.844792 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b34da3-1a0e-4077-b6be-3c26403560c2 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:53.878471 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5348fbe-7215-4dcf-a0b3-8ca6c3fe196a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:53.879269 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5348fbe-7215-4dcf-a0b3-8ca6c3fe196a None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:53.879557 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5348fbe-7215-4dcf-a0b3-8ca6c3fe196a None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:53.879839 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5348fbe-7215-4dcf-a0b3-8ca6c3fe196a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:53.889727 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5348fbe-7215-4dcf-a0b3-8ca6c3fe196a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:46:53.897309 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-e5348fbe-7215-4dcf-a0b3-8ca6c3fe196a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 10 12:46:53.897309 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:46:53.897309 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:46:53.897309 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:46:53.897309 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:46:53.897309 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:46:53.897309 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:46:53.897309 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:46:53.897309 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:46:53.897309 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:46:53.897309 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:46:53.897309 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:46:53.897309 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:46:53.897309 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:46:53.897309 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:46:53.897309 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:46:53.898421 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:46:53.898421 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:46:53.898421 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:46:53.898421 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:46:53.898421 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:46:53.898421 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:46:53.898421 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:46:53.898421 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:46:53.898421 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:46:53.898421 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:46:53.898421 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:46:53.898421 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:46:53.898421 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:46:53.898421 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:46:53.898421 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:46:53.898421 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:46:53.899343 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:46:53.899343 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:46:53.899343 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:46:53.899343 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:46:53.899343 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:46:53.899343 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:46:53.899343 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:46:53.899343 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:46:53.899343 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:46:53.899343 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:46:53.899343 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 10 12:46:53.899343 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:46:53.899343 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 601/1195] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Jun 10 12:46:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:46:53.904410 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9514343-b2bc-4b01-bdb3-29ec2e99c81d None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:53.978248 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:46:53.999884 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9514343-b2bc-4b01-bdb3-29ec2e99c81d None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:54.032918 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286292e6-d5b3-4b0f-a908-d65ea22d74df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:54.033575 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286292e6-d5b3-4b0f-a908-d65ea22d74df None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:54.033854 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286292e6-d5b3-4b0f-a908-d65ea22d74df None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:54.034377 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286292e6-d5b3-4b0f-a908-d65ea22d74df None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:54.035111 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286292e6-d5b3-4b0f-a908-d65ea22d74df None admin] RBAC: Adding query filter params (name=admin) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:46:54.035413 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286292e6-d5b3-4b0f-a908-d65ea22d74df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:54.036791 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286292e6-d5b3-4b0f-a908-d65ea22d74df None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:54.044567 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 595/1196] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Jun 10 12:46:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:46:54.050935 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5348fbe-7215-4dcf-a0b3-8ca6c3fe196a None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:54.146376 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5348fbe-7215-4dcf-a0b3-8ca6c3fe196a None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:54.179139 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f8d6ac-2ec3-4a55-b1a1-80f182d3b6e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:54.180026 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f8d6ac-2ec3-4a55-b1a1-80f182d3b6e1 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:54.180314 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f8d6ac-2ec3-4a55-b1a1-80f182d3b6e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:54.180590 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f8d6ac-2ec3-4a55-b1a1-80f182d3b6e1 None admin] PATH_INFO: `/v3/users/dbfccbed92dd456cbcdc7c73fac58bb7` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:54.181756 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f8d6ac-2ec3-4a55-b1a1-80f182d3b6e1 None admin] RBAC: Authorizing `identity:get_user(user_id=dbfccbed92dd456cbcdc7c73fac58bb7)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:54.183162 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f8d6ac-2ec3-4a55-b1a1-80f182d3b6e1 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:54.184790 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 602/1197] 10.222.0.27 () {56 vars in 1177 bytes} [Fri Jun 10 12:46:54 2022] GET /identity/v3/users/dbfccbed92dd456cbcdc7c73fac58bb7 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:46:54.190387 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286292e6-d5b3-4b0f-a908-d65ea22d74df None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:54.286608 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286292e6-d5b3-4b0f-a908-d65ea22d74df None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:54.320061 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0481e2b-36e1-4b50-aacb-05526e4d6095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:54.320714 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0481e2b-36e1-4b50-aacb-05526e4d6095 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:54.320992 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0481e2b-36e1-4b50-aacb-05526e4d6095 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:54.321264 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0481e2b-36e1-4b50-aacb-05526e4d6095 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:54.330569 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:46:54.386404 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0481e2b-36e1-4b50-aacb-05526e4d6095 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:54.387842 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0481e2b-36e1-4b50-aacb-05526e4d6095 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:54.481132 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 596/1198] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Jun 10 12:46:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 10 12:46:54.487034 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f8d6ac-2ec3-4a55-b1a1-80f182d3b6e1 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:54.489875 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:46:54.583969 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f8d6ac-2ec3-4a55-b1a1-80f182d3b6e1 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:54.617931 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd52f28f-5b2d-4afb-bc07-32739c8f8799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:54.618778 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd52f28f-5b2d-4afb-bc07-32739c8f8799 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:54.619056 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd52f28f-5b2d-4afb-bc07-32739c8f8799 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:54.619326 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd52f28f-5b2d-4afb-bc07-32739c8f8799 None admin] PATH_INFO: `/v3/projects/3ed6c933db4941b096e7cb7c9f37415e` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:54.620501 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd52f28f-5b2d-4afb-bc07-32739c8f8799 None admin] RBAC: Authorizing `identity:get_project(project_id=3ed6c933db4941b096e7cb7c9f37415e)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:54.621914 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd52f28f-5b2d-4afb-bc07-32739c8f8799 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:54.623593 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 603/1199] 10.222.0.27 () {56 vars in 1186 bytes} [Fri Jun 10 12:46:54 2022] GET /identity/v3/projects/3ed6c933db4941b096e7cb7c9f37415e => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:46:54.629594 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0481e2b-36e1-4b50-aacb-05526e4d6095 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:46:54.668314 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:46:54.725031 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0481e2b-36e1-4b50-aacb-05526e4d6095 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:46:54.757610 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2fc90b-a2e2-49c7-a01f-cefc95e34871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:46:54.758195 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2fc90b-a2e2-49c7-a01f-cefc95e34871 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:46:54.758475 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2fc90b-a2e2-49c7-a01f-cefc95e34871 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:46:54.758747 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2fc90b-a2e2-49c7-a01f-cefc95e34871 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:46:54.759568 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2fc90b-a2e2-49c7-a01f-cefc95e34871 None admin] RBAC: Adding query filter params (role.id=b43d5a8247e244869c2c70f986f42e8a, scope.project.id=3ed6c933db4941b096e7cb7c9f37415e, user.id=dbfccbed92dd456cbcdc7c73fac58bb7) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:46:54.759901 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2fc90b-a2e2-49c7-a01f-cefc95e34871 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:46:54.761310 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2fc90b-a2e2-49c7-a01f-cefc95e34871 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:46:54.767548 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 597/1200] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Jun 10 12:46:54 2022] GET /identity/v3/role_assignments?user.id=dbfccbed92dd456cbcdc7c73fac58bb7&scope.project.id=3ed6c933db4941b096e7cb7c9f37415e&role.id=b43d5a8247e244869c2c70f986f42e8a => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:47:04.613876 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=65 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:47:04.764334 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=49 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:48:05.944616 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b75bcf6-ff75-45ab-8b76-30cf48ef63bf None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:05.945197 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b75bcf6-ff75-45ab-8b76-30cf48ef63bf None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:05.945197 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b75bcf6-ff75-45ab-8b76-30cf48ef63bf None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:05.946637 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 604/1201] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:48:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:48:05.952031 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5d5db3-82f5-40e4-ba2c-2282b6fd4571 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:05.952497 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5d5db3-82f5-40e4-ba2c-2282b6fd4571 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:05.952599 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5d5db3-82f5-40e4-ba2c-2282b6fd4571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:05.969510 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-5d5d5db3-82f5-40e4-ba2c-2282b6fd4571 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:48:05.971828 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5d5d5db3-82f5-40e4-ba2c-2282b6fd4571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:48:06.034723 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 598/1202] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:48:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 10 12:48:06.060332 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b75bcf6-ff75-45ab-8b76-30cf48ef63bf None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:48:06.161189 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b75bcf6-ff75-45ab-8b76-30cf48ef63bf None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:48:06.193517 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a467b895-3aea-4374-b97f-dcf2da4dd36f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:48:06.194295 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a467b895-3aea-4374-b97f-dcf2da4dd36f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:06.194580 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a467b895-3aea-4374-b97f-dcf2da4dd36f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:06.194887 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a467b895-3aea-4374-b97f-dcf2da4dd36f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:06.205287 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a467b895-3aea-4374-b97f-dcf2da4dd36f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:48:06.207393 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a467b895-3aea-4374-b97f-dcf2da4dd36f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:48:06.224249 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-a467b895-3aea-4374-b97f-dcf2da4dd36f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:48:06.224249 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:48:06.224249 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:48:06.224249 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:48:06.224249 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:48:06.224249 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:48:06.224249 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:48:06.224249 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:48:06.224249 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:48:06.224249 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:48:06.224249 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:48:06.224249 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:48:06.224249 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 10 12:48:06.224249 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 10 12:48:06.224249 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 10 12:48:06.225380 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 10 12:48:06.225380 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:48:06.225380 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:48:06.225380 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:48:06.225380 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:48:06.225380 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:48:06.225380 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:48:06.225380 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:48:06.225380 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:48:06.225380 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:48:06.225380 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:48:06.225380 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:48:06.225380 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:48:06.225380 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:48:06.225380 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:48:06.225380 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:48:06.225380 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:48:06.226587 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 10 12:48:06.226587 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 10 12:48:06.226587 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:48:06.226587 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:48:06.226587 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 605/1203] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:48:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:48:06.231616 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5d5db3-82f5-40e4-ba2c-2282b6fd4571 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:48:06.326549 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5d5db3-82f5-40e4-ba2c-2282b6fd4571 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:48:06.354291 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:48:06.359882 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f741e65f-d6ad-4e62-bf6d-583e5a7efe90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:48:06.360545 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f741e65f-d6ad-4e62-bf6d-583e5a7efe90 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:06.360826 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f741e65f-d6ad-4e62-bf6d-583e5a7efe90 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:06.361099 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f741e65f-d6ad-4e62-bf6d-583e5a7efe90 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:06.361817 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f741e65f-d6ad-4e62-bf6d-583e5a7efe90 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:48:06.362117 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f741e65f-d6ad-4e62-bf6d-583e5a7efe90 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:48:06.363514 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f741e65f-d6ad-4e62-bf6d-583e5a7efe90 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:48:06.375721 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 599/1204] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Jun 10 12:48:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:48:06.380242 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a467b895-3aea-4374-b97f-dcf2da4dd36f None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:48:06.432281 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:48:06.477085 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a467b895-3aea-4374-b97f-dcf2da4dd36f None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:48:06.509335 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23edb380-b2f4-4777-988e-d1f176a6904d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:48:06.509987 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23edb380-b2f4-4777-988e-d1f176a6904d None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:06.510269 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23edb380-b2f4-4777-988e-d1f176a6904d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:06.510544 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23edb380-b2f4-4777-988e-d1f176a6904d None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:06.511230 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23edb380-b2f4-4777-988e-d1f176a6904d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:48:06.512750 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23edb380-b2f4-4777-988e-d1f176a6904d None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:48:06.598830 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 606/1205] 10.222.0.27 () {60 vars in 1132 bytes} [Fri Jun 10 12:48:06 2022] POST /identity/v3/users => generated 251 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 12:48:08.754498 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca8298d-1ffa-4407-b914-a4808a1cf845 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:08.755065 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca8298d-1ffa-4407-b914-a4808a1cf845 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:08.755065 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca8298d-1ffa-4407-b914-a4808a1cf845 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:08.756464 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 600/1206] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:48:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:48:08.761636 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230ff354-c9ec-4fac-b47a-b5ebbebcc772 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:08.762052 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230ff354-c9ec-4fac-b47a-b5ebbebcc772 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:08.762199 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230ff354-c9ec-4fac-b47a-b5ebbebcc772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:08.778471 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-230ff354-c9ec-4fac-b47a-b5ebbebcc772 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:48:08.779643 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-230ff354-c9ec-4fac-b47a-b5ebbebcc772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:48:08.844436 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 607/1207] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:48:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 10 12:48:08.868138 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca8298d-1ffa-4407-b914-a4808a1cf845 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:48:08.968899 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca8298d-1ffa-4407-b914-a4808a1cf845 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:48:09.000728 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387612d4-64a5-4f5c-ba49-05576f269187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:48:09.001507 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387612d4-64a5-4f5c-ba49-05576f269187 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:09.001785 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387612d4-64a5-4f5c-ba49-05576f269187 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:09.002056 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387612d4-64a5-4f5c-ba49-05576f269187 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:09.012308 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387612d4-64a5-4f5c-ba49-05576f269187 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:48:09.019813 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-387612d4-64a5-4f5c-ba49-05576f269187 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 10 12:48:09.019813 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:48:09.019813 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:48:09.019813 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:48:09.019813 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:48:09.019813 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:48:09.019813 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:48:09.019813 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:48:09.019813 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:48:09.019813 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:48:09.019813 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:48:09.019813 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:48:09.019813 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:48:09.019813 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:48:09.019813 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:48:09.019813 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:48:09.020942 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:48:09.020942 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:48:09.020942 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:48:09.020942 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:48:09.020942 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:48:09.020942 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:48:09.020942 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:48:09.020942 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:48:09.020942 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:48:09.020942 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:48:09.020942 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:48:09.020942 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:48:09.020942 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:48:09.020942 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:48:09.020942 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:48:09.020942 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:48:09.021902 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:48:09.021902 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:48:09.021902 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:48:09.021902 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:48:09.021902 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:48:09.021902 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:48:09.021902 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:48:09.021902 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:48:09.021902 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:48:09.021902 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:48:09.021902 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 10 12:48:09.021902 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:48:09.021902 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 601/1208] 10.222.0.27 () {56 vars in 1102 bytes} [Fri Jun 10 12:48:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:48:09.027386 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230ff354-c9ec-4fac-b47a-b5ebbebcc772 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:48:09.067135 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:48:09.123159 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230ff354-c9ec-4fac-b47a-b5ebbebcc772 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:48:09.155396 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11741c36-bc07-4306-a087-669a58028f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:48:09.156832 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11741c36-bc07-4306-a087-669a58028f16 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:09.157114 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11741c36-bc07-4306-a087-669a58028f16 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:09.157386 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11741c36-bc07-4306-a087-669a58028f16 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:09.158131 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11741c36-bc07-4306-a087-669a58028f16 None admin] RBAC: Adding query filter params (name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:48:09.158446 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11741c36-bc07-4306-a087-669a58028f16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:48:09.159799 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11741c36-bc07-4306-a087-669a58028f16 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:48:09.167615 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 608/1209] 10.222.0.27 () {56 vars in 1103 bytes} [Fri Jun 10 12:48:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:48:09.174131 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387612d4-64a5-4f5c-ba49-05576f269187 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:48:09.223242 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:48:09.271058 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387612d4-64a5-4f5c-ba49-05576f269187 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:48:09.304216 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22589475-3a26-44a9-b65a-5b0944e486b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:48:09.305000 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22589475-3a26-44a9-b65a-5b0944e486b8 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:09.305282 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22589475-3a26-44a9-b65a-5b0944e486b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:09.305556 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22589475-3a26-44a9-b65a-5b0944e486b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:09.315201 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22589475-3a26-44a9-b65a-5b0944e486b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:48:09.316663 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22589475-3a26-44a9-b65a-5b0944e486b8 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:48:09.333928 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-22589475-3a26-44a9-b65a-5b0944e486b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:48:09.333928 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:48:09.333928 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:48:09.333928 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:48:09.333928 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:48:09.333928 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:48:09.333928 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:48:09.333928 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:48:09.333928 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:48:09.333928 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:48:09.333928 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:48:09.333928 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:48:09.333928 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 10 12:48:09.333928 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 10 12:48:09.333928 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 10 12:48:09.334973 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 10 12:48:09.334973 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:48:09.334973 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:48:09.334973 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:48:09.334973 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:48:09.334973 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:48:09.334973 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:48:09.334973 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:48:09.334973 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:48:09.334973 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:48:09.334973 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:48:09.334973 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:48:09.334973 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:48:09.334973 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:48:09.334973 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:48:09.334973 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:48:09.334973 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:48:09.335965 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 10 12:48:09.335965 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 10 12:48:09.335965 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:48:09.335965 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:48:09.335965 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 602/1210] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:48:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:48:09.340770 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11741c36-bc07-4306-a087-669a58028f16 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:48:09.435518 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:48:09.438306 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11741c36-bc07-4306-a087-669a58028f16 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:48:09.470562 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ad40a3-8fa2-4d71-a315-e90e23e2aa9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:48:09.471213 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ad40a3-8fa2-4d71-a315-e90e23e2aa9c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:09.471498 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ad40a3-8fa2-4d71-a315-e90e23e2aa9c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:09.471945 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ad40a3-8fa2-4d71-a315-e90e23e2aa9c None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:09.472600 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ad40a3-8fa2-4d71-a315-e90e23e2aa9c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:48:09.472924 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ad40a3-8fa2-4d71-a315-e90e23e2aa9c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:48:09.474245 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ad40a3-8fa2-4d71-a315-e90e23e2aa9c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:48:09.485389 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 609/1211] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Jun 10 12:48:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:48:09.491043 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22589475-3a26-44a9-b65a-5b0944e486b8 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:48:09.591088 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22589475-3a26-44a9-b65a-5b0944e486b8 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:48:09.592605 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:48:09.624438 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf72bf1-ffd3-4f27-983b-97917348de32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:48:09.625738 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf72bf1-ffd3-4f27-983b-97917348de32 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:09.626027 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf72bf1-ffd3-4f27-983b-97917348de32 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:09.626396 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf72bf1-ffd3-4f27-983b-97917348de32 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:09.633224 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf72bf1-ffd3-4f27-983b-97917348de32 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:48:09.634666 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf72bf1-ffd3-4f27-983b-97917348de32 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:48:09.649538 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-fdf72bf1-ffd3-4f27-983b-97917348de32 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 10 12:48:09.649538 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:48:09.649538 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:48:09.649538 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:48:09.649538 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:48:09.649538 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:48:09.649538 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:48:09.649538 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:48:09.649538 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:48:09.649538 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:48:09.649538 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:48:09.649538 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:48:09.649538 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 10 12:48:09.649538 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 10 12:48:09.649538 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 10 12:48:09.649538 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 10 12:48:09.650693 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:48:09.650693 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:48:09.650693 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 10 12:48:09.650693 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 10 12:48:09.650693 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 10 12:48:09.650693 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 10 12:48:09.650693 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:48:09.650693 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:48:09.650693 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:48:09.650693 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:48:09.650693 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:48:09.650693 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:48:09.650693 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:48:09.650693 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:48:09.650693 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:48:09.650693 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:48:09.650693 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:48:09.650693 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:48:09.651949 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:48:09.651949 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:48:09.651949 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 10 12:48:09.651949 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 10 12:48:09.651949 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 10 12:48:09.651949 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 10 12:48:09.651949 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 10 12:48:09.651949 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 10 12:48:09.651949 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 10 12:48:09.651949 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:48:09.651949 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 603/1212] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:48:09 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:48:09.656820 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ad40a3-8fa2-4d71-a315-e90e23e2aa9c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:48:09.755744 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ad40a3-8fa2-4d71-a315-e90e23e2aa9c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:48:09.789460 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a697b1d5-dde3-4351-a9e9-8ca47cfa664c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:48:09.790125 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a697b1d5-dde3-4351-a9e9-8ca47cfa664c None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:09.790409 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a697b1d5-dde3-4351-a9e9-8ca47cfa664c None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:09.790687 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a697b1d5-dde3-4351-a9e9-8ca47cfa664c None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:09.791543 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a697b1d5-dde3-4351-a9e9-8ca47cfa664c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:48:09.791900 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a697b1d5-dde3-4351-a9e9-8ca47cfa664c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:48:09.793325 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a697b1d5-dde3-4351-a9e9-8ca47cfa664c None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:48:09.809893 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 610/1213] 10.222.0.27 () {56 vars in 1143 bytes} [Fri Jun 10 12:48:09 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:48:09.816586 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf72bf1-ffd3-4f27-983b-97917348de32 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:48:09.918985 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf72bf1-ffd3-4f27-983b-97917348de32 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:48:09.952243 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e764e8f-7eab-40e7-96dd-6476c53061d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:48:09.953450 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e764e8f-7eab-40e7-96dd-6476c53061d2 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:09.953732 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e764e8f-7eab-40e7-96dd-6476c53061d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:09.954028 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e764e8f-7eab-40e7-96dd-6476c53061d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:09.982404 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:48:10.019309 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e764e8f-7eab-40e7-96dd-6476c53061d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:48:10.020823 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e764e8f-7eab-40e7-96dd-6476c53061d2 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:48:10.114114 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 604/1214] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Jun 10 12:48:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 10 12:48:10.120297 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a697b1d5-dde3-4351-a9e9-8ca47cfa664c None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:48:10.140141 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:48:10.216491 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a697b1d5-dde3-4351-a9e9-8ca47cfa664c None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:48:10.250246 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce87a46-2f7c-4c99-9349-241eb87acb6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:48:10.251032 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce87a46-2f7c-4c99-9349-241eb87acb6b None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:10.251308 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce87a46-2f7c-4c99-9349-241eb87acb6b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:10.251580 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce87a46-2f7c-4c99-9349-241eb87acb6b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:10.261163 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce87a46-2f7c-4c99-9349-241eb87acb6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:48:10.262581 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce87a46-2f7c-4c99-9349-241eb87acb6b None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:48:10.279268 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-dce87a46-2f7c-4c99-9349-241eb87acb6b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:48:10.279268 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:48:10.279268 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:48:10.279268 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:48:10.279268 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:48:10.279268 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:48:10.279268 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:48:10.279268 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:48:10.279268 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:48:10.279268 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:48:10.279268 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:48:10.279268 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:48:10.279268 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 10 12:48:10.279268 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 10 12:48:10.279268 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 10 12:48:10.280403 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 10 12:48:10.280403 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:48:10.280403 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:48:10.280403 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:48:10.280403 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:48:10.280403 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:48:10.280403 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:48:10.280403 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:48:10.280403 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:48:10.280403 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:48:10.280403 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:48:10.280403 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:48:10.280403 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:48:10.280403 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:48:10.280403 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:48:10.280403 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:48:10.280403 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:48:10.281796 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 10 12:48:10.281796 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 10 12:48:10.281796 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:48:10.281796 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:48:10.281796 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 611/1215] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:48:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:48:10.286386 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e764e8f-7eab-40e7-96dd-6476c53061d2 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:48:10.345007 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:48:10.382791 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e764e8f-7eab-40e7-96dd-6476c53061d2 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:48:10.415380 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6882dd1-71bc-41cb-8433-ccbcfde704fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:48:10.416059 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6882dd1-71bc-41cb-8433-ccbcfde704fc None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:10.416342 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6882dd1-71bc-41cb-8433-ccbcfde704fc None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:10.416618 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6882dd1-71bc-41cb-8433-ccbcfde704fc None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:10.417330 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6882dd1-71bc-41cb-8433-ccbcfde704fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:48:10.417633 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6882dd1-71bc-41cb-8433-ccbcfde704fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:48:10.419008 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6882dd1-71bc-41cb-8433-ccbcfde704fc None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:48:10.430938 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 605/1216] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Jun 10 12:48:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:48:10.436645 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce87a46-2f7c-4c99-9349-241eb87acb6b None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:48:10.504532 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:48:10.534112 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce87a46-2f7c-4c99-9349-241eb87acb6b None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:48:10.566324 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bf2c97-4ab8-4e34-9481-3b77d0819246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:48:10.567138 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bf2c97-4ab8-4e34-9481-3b77d0819246 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:10.567421 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bf2c97-4ab8-4e34-9481-3b77d0819246 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:10.567696 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bf2c97-4ab8-4e34-9481-3b77d0819246 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:10.574182 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bf2c97-4ab8-4e34-9481-3b77d0819246 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:48:10.575648 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bf2c97-4ab8-4e34-9481-3b77d0819246 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:48:10.590227 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-59bf2c97-4ab8-4e34-9481-3b77d0819246 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 10 12:48:10.590227 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:48:10.590227 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:48:10.590227 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:48:10.590227 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:48:10.590227 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:48:10.590227 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:48:10.590227 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:48:10.590227 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:48:10.590227 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:48:10.590227 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:48:10.590227 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:48:10.590227 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 10 12:48:10.590227 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 10 12:48:10.590227 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 10 12:48:10.595835 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 10 12:48:10.595835 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:48:10.595835 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:48:10.595835 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:48:10.595835 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:48:10.595835 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:48:10.595835 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:48:10.595835 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:48:10.595835 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:48:10.595835 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:48:10.595835 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:48:10.595835 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:48:10.595835 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:48:10.595835 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:48:10.595835 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:48:10.595835 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:48:10.595835 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:48:10.596807 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 10 12:48:10.596807 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 10 12:48:10.596807 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 10 12:48:10.596807 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 10 12:48:10.596807 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 10 12:48:10.596807 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 10 12:48:10.596807 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 10 12:48:10.596807 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:48:10.596807 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 612/1217] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Jun 10 12:48:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:48:10.597357 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6882dd1-71bc-41cb-8433-ccbcfde704fc None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:48:10.692888 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6882dd1-71bc-41cb-8433-ccbcfde704fc None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:48:10.714385 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:48:10.725780 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4593f07e-1de9-4b2f-acbb-13b26fb270c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:48:10.726426 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4593f07e-1de9-4b2f-acbb-13b26fb270c2 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:10.726707 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4593f07e-1de9-4b2f-acbb-13b26fb270c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:10.726977 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4593f07e-1de9-4b2f-acbb-13b26fb270c2 None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:10.727751 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4593f07e-1de9-4b2f-acbb-13b26fb270c2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:48:10.728086 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4593f07e-1de9-4b2f-acbb-13b26fb270c2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:48:10.729560 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4593f07e-1de9-4b2f-acbb-13b26fb270c2 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:48:10.741258 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 606/1218] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Jun 10 12:48:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:48:10.747477 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bf2c97-4ab8-4e34-9481-3b77d0819246 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:48:10.845630 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bf2c97-4ab8-4e34-9481-3b77d0819246 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:48:10.874922 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:48:10.880687 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89766487-1cc5-4fcc-973b-f0e66ca85240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:48:10.881338 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89766487-1cc5-4fcc-973b-f0e66ca85240 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:10.883464 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89766487-1cc5-4fcc-973b-f0e66ca85240 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:10.883464 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89766487-1cc5-4fcc-973b-f0e66ca85240 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:10.883464 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89766487-1cc5-4fcc-973b-f0e66ca85240 None admin] RBAC: Adding query filter params (role.id=ea7e8f74523b4322857fb3f4307d171e, scope.project.id=73ea5b23a2d34544ab493f4c5e4034b8, user.id=d972ab782b3f4d2aba575314f16dafc7) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:48:10.883464 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89766487-1cc5-4fcc-973b-f0e66ca85240 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:48:10.884439 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89766487-1cc5-4fcc-973b-f0e66ca85240 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:48:10.889663 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 613/1219] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Jun 10 12:48:10 2022] GET /identity/v3/role_assignments?user.id=d972ab782b3f4d2aba575314f16dafc7&scope.project.id=73ea5b23a2d34544ab493f4c5e4034b8&role.id=ea7e8f74523b4322857fb3f4307d171e => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:48:12.969367 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfa4eb6-7e93-4b27-8b18-96cb602220d6 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:12.969918 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfa4eb6-7e93-4b27-8b18-96cb602220d6 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:12.970020 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfa4eb6-7e93-4b27-8b18-96cb602220d6 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:12.971469 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 607/1220] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:48:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:48:12.978018 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d897300e-d111-4bb6-a5eb-67ac389e8de9 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:12.978430 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d897300e-d111-4bb6-a5eb-67ac389e8de9 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:12.978579 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d897300e-d111-4bb6-a5eb-67ac389e8de9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:12.995581 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-d897300e-d111-4bb6-a5eb-67ac389e8de9 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:48:12.996814 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d897300e-d111-4bb6-a5eb-67ac389e8de9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:48:13.061041 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 614/1221] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:48:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 10 12:48:13.084554 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfa4eb6-7e93-4b27-8b18-96cb602220d6 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:48:13.184355 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfa4eb6-7e93-4b27-8b18-96cb602220d6 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:48:13.217772 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d8c0c5-ae5f-430f-a8d1-4c66b084cc55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:48:13.218606 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d8c0c5-ae5f-430f-a8d1-4c66b084cc55 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:13.218906 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d8c0c5-ae5f-430f-a8d1-4c66b084cc55 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:13.219179 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d8c0c5-ae5f-430f-a8d1-4c66b084cc55 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:13.229364 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d8c0c5-ae5f-430f-a8d1-4c66b084cc55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:48:13.237716 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-c7d8c0c5-ae5f-430f-a8d1-4c66b084cc55 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 10 12:48:13.237716 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:48:13.237716 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:48:13.237716 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:48:13.237716 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:48:13.237716 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:48:13.237716 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:48:13.237716 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:48:13.237716 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:48:13.237716 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:48:13.237716 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:48:13.237716 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:48:13.237716 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:48:13.237716 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:48:13.237716 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:48:13.237716 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:48:13.238870 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:48:13.238870 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:48:13.238870 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:48:13.238870 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:48:13.238870 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:48:13.238870 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:48:13.238870 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:48:13.238870 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:48:13.238870 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:48:13.238870 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:48:13.238870 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:48:13.238870 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:48:13.238870 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:48:13.238870 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:48:13.238870 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:48:13.238870 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:48:13.240083 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:48:13.240083 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:48:13.240083 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:48:13.240083 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:48:13.240083 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:48:13.240083 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:48:13.240083 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:48:13.240083 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:48:13.240083 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:48:13.240083 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:48:13.240083 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 10 12:48:13.240083 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:48:13.240083 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 608/1222] 10.222.0.27 () {56 vars in 1102 bytes} [Fri Jun 10 12:48:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:48:13.244899 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d897300e-d111-4bb6-a5eb-67ac389e8de9 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:48:13.345306 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d897300e-d111-4bb6-a5eb-67ac389e8de9 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:48:13.368144 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:48:13.378948 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d1c837-b9cf-4e7a-a92e-db33903aca37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:48:13.379610 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d1c837-b9cf-4e7a-a92e-db33903aca37 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:13.379963 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d1c837-b9cf-4e7a-a92e-db33903aca37 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:13.380243 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d1c837-b9cf-4e7a-a92e-db33903aca37 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:13.381139 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d1c837-b9cf-4e7a-a92e-db33903aca37 None admin] RBAC: Adding query filter params (name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:48:13.381646 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d1c837-b9cf-4e7a-a92e-db33903aca37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:48:13.384155 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d1c837-b9cf-4e7a-a92e-db33903aca37 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:48:13.393796 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 615/1223] 10.222.0.27 () {56 vars in 1103 bytes} [Fri Jun 10 12:48:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:48:13.399818 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d8c0c5-ae5f-430f-a8d1-4c66b084cc55 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:48:13.443158 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:48:13.498390 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d8c0c5-ae5f-430f-a8d1-4c66b084cc55 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:48:13.531392 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c386e1-6a13-4ebc-a769-6365665344d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:48:13.532200 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c386e1-6a13-4ebc-a769-6365665344d2 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:13.532482 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c386e1-6a13-4ebc-a769-6365665344d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:13.532754 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c386e1-6a13-4ebc-a769-6365665344d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:13.542858 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c386e1-6a13-4ebc-a769-6365665344d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:48:13.544321 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c386e1-6a13-4ebc-a769-6365665344d2 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:48:13.561647 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-b0c386e1-6a13-4ebc-a769-6365665344d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:48:13.561647 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:48:13.561647 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:48:13.561647 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:48:13.561647 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:48:13.561647 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:48:13.561647 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:48:13.561647 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:48:13.561647 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:48:13.561647 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:48:13.561647 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:48:13.561647 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:48:13.561647 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 10 12:48:13.561647 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 10 12:48:13.561647 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 10 12:48:13.562731 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 10 12:48:13.562731 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:48:13.562731 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:48:13.562731 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:48:13.562731 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:48:13.562731 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:48:13.562731 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:48:13.562731 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:48:13.562731 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:48:13.562731 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:48:13.562731 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:48:13.562731 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:48:13.562731 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:48:13.562731 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:48:13.562731 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:48:13.562731 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:48:13.562731 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:48:13.564126 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 10 12:48:13.564126 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 10 12:48:13.564126 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:48:13.564126 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:48:13.564126 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 609/1224] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:48:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:48:13.568662 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d1c837-b9cf-4e7a-a92e-db33903aca37 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:48:13.669450 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d1c837-b9cf-4e7a-a92e-db33903aca37 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:48:13.701888 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c5fdd8-3157-4d96-b0db-51600018bd8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:48:13.702503 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c5fdd8-3157-4d96-b0db-51600018bd8d None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:13.702786 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c5fdd8-3157-4d96-b0db-51600018bd8d None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:13.703061 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c5fdd8-3157-4d96-b0db-51600018bd8d None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:13.703780 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c5fdd8-3157-4d96-b0db-51600018bd8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:48:13.704113 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c5fdd8-3157-4d96-b0db-51600018bd8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:48:13.705471 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c5fdd8-3157-4d96-b0db-51600018bd8d None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:48:13.716741 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 616/1225] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Jun 10 12:48:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:48:13.723021 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c386e1-6a13-4ebc-a769-6365665344d2 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:48:13.818327 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:48:13.824893 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c386e1-6a13-4ebc-a769-6365665344d2 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:48:13.858212 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56411e56-7333-4697-b402-580ed00423ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:48:13.859008 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56411e56-7333-4697-b402-580ed00423ad None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:13.859289 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56411e56-7333-4697-b402-580ed00423ad None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:13.859563 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56411e56-7333-4697-b402-580ed00423ad None admin] PATH_INFO: `/v3/users/placement` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:13.866736 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56411e56-7333-4697-b402-580ed00423ad None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:48:13.868205 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56411e56-7333-4697-b402-580ed00423ad None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:48:13.882711 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-56411e56-7333-4697-b402-580ed00423ad None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 10 12:48:13.882711 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:48:13.882711 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:48:13.882711 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:48:13.882711 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:48:13.882711 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:48:13.882711 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:48:13.882711 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:48:13.882711 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:48:13.882711 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:48:13.882711 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:48:13.882711 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:48:13.882711 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 10 12:48:13.882711 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 10 12:48:13.882711 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 10 12:48:13.882711 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 10 12:48:13.883904 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:48:13.883904 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:48:13.883904 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 10 12:48:13.883904 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 10 12:48:13.883904 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 10 12:48:13.883904 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 10 12:48:13.883904 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:48:13.883904 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:48:13.883904 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:48:13.883904 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:48:13.883904 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:48:13.883904 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:48:13.883904 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:48:13.883904 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:48:13.883904 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:48:13.883904 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:48:13.883904 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:48:13.883904 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:48:13.885224 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:48:13.885224 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:48:13.885224 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 10 12:48:13.885224 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 10 12:48:13.885224 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 10 12:48:13.885224 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 10 12:48:13.885224 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 10 12:48:13.885224 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 10 12:48:13.885224 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 10 12:48:13.885224 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:48:13.885224 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 610/1226] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:48:13 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:48:13.889737 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c5fdd8-3157-4d96-b0db-51600018bd8d None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:48:13.938969 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:48:13.991028 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c5fdd8-3157-4d96-b0db-51600018bd8d None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:48:14.024166 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04b2b53-2aef-4ded-8e9f-980c456717ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:48:14.024873 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04b2b53-2aef-4ded-8e9f-980c456717ff None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:14.025175 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04b2b53-2aef-4ded-8e9f-980c456717ff None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:14.025450 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04b2b53-2aef-4ded-8e9f-980c456717ff None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:14.026453 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04b2b53-2aef-4ded-8e9f-980c456717ff None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:48:14.026781 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04b2b53-2aef-4ded-8e9f-980c456717ff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:48:14.028302 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04b2b53-2aef-4ded-8e9f-980c456717ff None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:48:14.047073 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 617/1227] 10.222.0.27 () {56 vars in 1143 bytes} [Fri Jun 10 12:48:13 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:48:14.053127 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56411e56-7333-4697-b402-580ed00423ad None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:48:14.151622 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56411e56-7333-4697-b402-580ed00423ad None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:48:14.184926 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f023bc-13e8-43cc-8b64-e19d0fe3329c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:48:14.185702 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f023bc-13e8-43cc-8b64-e19d0fe3329c None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:14.185988 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f023bc-13e8-43cc-8b64-e19d0fe3329c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:14.186275 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f023bc-13e8-43cc-8b64-e19d0fe3329c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:14.196639 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f023bc-13e8-43cc-8b64-e19d0fe3329c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:48:14.198105 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f023bc-13e8-43cc-8b64-e19d0fe3329c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:48:14.216030 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-07f023bc-13e8-43cc-8b64-e19d0fe3329c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:48:14.216030 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:48:14.216030 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:48:14.216030 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:48:14.216030 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:48:14.216030 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:48:14.216030 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:48:14.216030 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:48:14.216030 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:48:14.216030 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:48:14.216030 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:48:14.216030 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:48:14.216030 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 10 12:48:14.216030 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 10 12:48:14.216030 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 10 12:48:14.219386 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 10 12:48:14.219386 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:48:14.219386 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:48:14.219386 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:48:14.219386 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:48:14.219386 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:48:14.219386 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:48:14.219386 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:48:14.219386 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:48:14.219386 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:48:14.219386 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:48:14.219386 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:48:14.219386 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:48:14.219386 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:48:14.219386 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:48:14.219386 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:48:14.219386 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:48:14.220385 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 10 12:48:14.220385 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 10 12:48:14.220385 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:48:14.220385 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:48:14.220385 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 611/1228] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:48:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:48:14.222981 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04b2b53-2aef-4ded-8e9f-980c456717ff None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:48:14.319783 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04b2b53-2aef-4ded-8e9f-980c456717ff None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:48:14.322012 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:48:14.353013 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1bcc21-e904-4975-8cd1-184bc3e649bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:48:14.353730 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1bcc21-e904-4975-8cd1-184bc3e649bf None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:14.354017 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1bcc21-e904-4975-8cd1-184bc3e649bf None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:14.354238 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1bcc21-e904-4975-8cd1-184bc3e649bf None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:14.354958 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1bcc21-e904-4975-8cd1-184bc3e649bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:48:14.355267 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1bcc21-e904-4975-8cd1-184bc3e649bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:48:14.356649 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1bcc21-e904-4975-8cd1-184bc3e649bf None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:48:14.367410 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 618/1229] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Jun 10 12:48:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:48:14.373067 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f023bc-13e8-43cc-8b64-e19d0fe3329c None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:48:14.382234 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:48:14.470756 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f023bc-13e8-43cc-8b64-e19d0fe3329c None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:48:14.505379 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c32e92-051b-4c2e-bf24-592a19014d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:48:14.506208 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c32e92-051b-4c2e-bf24-592a19014d0b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:14.506489 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c32e92-051b-4c2e-bf24-592a19014d0b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:14.506765 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c32e92-051b-4c2e-bf24-592a19014d0b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:14.513272 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c32e92-051b-4c2e-bf24-592a19014d0b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:48:14.514706 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c32e92-051b-4c2e-bf24-592a19014d0b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:48:14.528265 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-e4c32e92-051b-4c2e-bf24-592a19014d0b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 10 12:48:14.528265 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:48:14.528265 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:48:14.528265 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:48:14.528265 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:48:14.528265 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:48:14.528265 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:48:14.528265 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:48:14.528265 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:48:14.528265 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:48:14.528265 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:48:14.528265 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:48:14.528265 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 10 12:48:14.528265 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 10 12:48:14.528265 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 10 12:48:14.529346 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 10 12:48:14.529346 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:48:14.529346 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:48:14.529346 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:48:14.529346 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:48:14.529346 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:48:14.529346 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:48:14.529346 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:48:14.529346 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:48:14.529346 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:48:14.529346 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:48:14.529346 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:48:14.529346 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:48:14.529346 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:48:14.529346 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:48:14.529346 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:48:14.529346 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:48:14.530608 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 10 12:48:14.530608 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 10 12:48:14.530608 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 10 12:48:14.530608 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 10 12:48:14.530608 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 10 12:48:14.530608 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 10 12:48:14.530608 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 10 12:48:14.530608 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:48:14.530608 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 612/1230] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Jun 10 12:48:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:48:14.535534 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1bcc21-e904-4975-8cd1-184bc3e649bf None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:48:14.633223 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1bcc21-e904-4975-8cd1-184bc3e649bf None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:48:14.665277 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64fa41e-b157-41e0-b616-4c2803885cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:48:14.665906 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64fa41e-b157-41e0-b616-4c2803885cd9 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:14.666190 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64fa41e-b157-41e0-b616-4c2803885cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:14.666460 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64fa41e-b157-41e0-b616-4c2803885cd9 None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:14.667231 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64fa41e-b157-41e0-b616-4c2803885cd9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:48:14.667534 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64fa41e-b157-41e0-b616-4c2803885cd9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:48:14.668981 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64fa41e-b157-41e0-b616-4c2803885cd9 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:48:14.681923 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 619/1231] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Jun 10 12:48:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:48:14.688452 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c32e92-051b-4c2e-bf24-592a19014d0b None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:48:14.753374 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:48:14.785898 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c32e92-051b-4c2e-bf24-592a19014d0b None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:48:14.818379 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6731a31b-6d88-452a-a118-1d2866c65f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:48:14.819248 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6731a31b-6d88-452a-a118-1d2866c65f61 None admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:14.819530 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6731a31b-6d88-452a-a118-1d2866c65f61 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:14.820057 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6731a31b-6d88-452a-a118-1d2866c65f61 None admin] PATH_INFO: `/v3/projects/73ea5b23a2d34544ab493f4c5e4034b8/users/d972ab782b3f4d2aba575314f16dafc7/roles/ea7e8f74523b4322857fb3f4307d171e` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:14.834196 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6731a31b-6d88-452a-a118-1d2866c65f61 None admin] RBAC: Authorizing `identity:create_grant(project_id=73ea5b23a2d34544ab493f4c5e4034b8, user_id=d972ab782b3f4d2aba575314f16dafc7, role_id=ea7e8f74523b4322857fb3f4307d171e)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:48:14.835635 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6731a31b-6d88-452a-a118-1d2866c65f61 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:48:14.913315 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 613/1232] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Jun 10 12:48:14 2022] PUT /identity/v3/projects/73ea5b23a2d34544ab493f4c5e4034b8/users/d972ab782b3f4d2aba575314f16dafc7/roles/ea7e8f74523b4322857fb3f4307d171e => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 12:48:16.982374 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b20d8a4-be5b-4c6e-ba55-ed615a5917f8 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:16.982916 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b20d8a4-be5b-4c6e-ba55-ed615a5917f8 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:16.982916 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b20d8a4-be5b-4c6e-ba55-ed615a5917f8 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:16.984336 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 620/1233] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:48:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:48:16.989529 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88851ebc-158c-4f0a-80e8-63725fd01da6 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:16.989996 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88851ebc-158c-4f0a-80e8-63725fd01da6 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:16.990118 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88851ebc-158c-4f0a-80e8-63725fd01da6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:17.007174 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-88851ebc-158c-4f0a-80e8-63725fd01da6 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:48:17.008389 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-88851ebc-158c-4f0a-80e8-63725fd01da6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:48:17.073186 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 614/1234] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:48:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 10 12:48:17.097727 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b20d8a4-be5b-4c6e-ba55-ed615a5917f8 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:48:17.125807 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:48:17.196484 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b20d8a4-be5b-4c6e-ba55-ed615a5917f8 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:48:17.229411 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69cab32-e19b-426c-af0a-2dc05422d3c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:48:17.230230 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69cab32-e19b-426c-af0a-2dc05422d3c6 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:17.230506 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69cab32-e19b-426c-af0a-2dc05422d3c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:17.230775 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69cab32-e19b-426c-af0a-2dc05422d3c6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:17.240705 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69cab32-e19b-426c-af0a-2dc05422d3c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:48:17.248316 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-d69cab32-e19b-426c-af0a-2dc05422d3c6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 10 12:48:17.248316 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:48:17.248316 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:48:17.248316 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:48:17.248316 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:48:17.248316 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:48:17.248316 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:48:17.248316 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:48:17.248316 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:48:17.248316 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:48:17.248316 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:48:17.248316 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:48:17.248316 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:48:17.248316 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:48:17.248316 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:48:17.248316 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:48:17.250015 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:48:17.250015 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:48:17.250015 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:48:17.250015 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:48:17.250015 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:48:17.250015 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:48:17.250015 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:48:17.250015 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:48:17.250015 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:48:17.250015 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:48:17.250015 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:48:17.250015 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:48:17.250015 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:48:17.250015 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:48:17.250015 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:48:17.250015 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:48:17.250950 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:48:17.250950 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:48:17.250950 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:48:17.250950 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:48:17.250950 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:48:17.250950 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:48:17.250950 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:48:17.250950 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:48:17.250950 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:48:17.250950 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:48:17.250950 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 10 12:48:17.250950 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:48:17.250950 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 621/1235] 10.222.0.27 () {56 vars in 1102 bytes} [Fri Jun 10 12:48:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:48:17.256390 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88851ebc-158c-4f0a-80e8-63725fd01da6 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:48:17.302377 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:48:17.352919 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88851ebc-158c-4f0a-80e8-63725fd01da6 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:48:17.390292 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d105f6d2-2549-4f72-b1c2-9447dac2c956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:48:17.390894 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d105f6d2-2549-4f72-b1c2-9447dac2c956 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:17.391211 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d105f6d2-2549-4f72-b1c2-9447dac2c956 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:17.391487 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d105f6d2-2549-4f72-b1c2-9447dac2c956 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:17.392271 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d105f6d2-2549-4f72-b1c2-9447dac2c956 None admin] RBAC: Adding query filter params (name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:48:17.392577 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d105f6d2-2549-4f72-b1c2-9447dac2c956 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:48:17.393944 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d105f6d2-2549-4f72-b1c2-9447dac2c956 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:48:17.401821 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 615/1236] 10.222.0.27 () {56 vars in 1103 bytes} [Fri Jun 10 12:48:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:48:17.407893 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69cab32-e19b-426c-af0a-2dc05422d3c6 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:48:17.491289 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:48:17.507221 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69cab32-e19b-426c-af0a-2dc05422d3c6 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:48:17.539603 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f56787-a9e5-49bd-827d-efc550b76765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:48:17.540421 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f56787-a9e5-49bd-827d-efc550b76765 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:17.540706 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f56787-a9e5-49bd-827d-efc550b76765 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:17.540986 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f56787-a9e5-49bd-827d-efc550b76765 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:17.551622 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f56787-a9e5-49bd-827d-efc550b76765 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:48:17.553095 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f56787-a9e5-49bd-827d-efc550b76765 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:48:17.570726 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-b1f56787-a9e5-49bd-827d-efc550b76765 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:48:17.570726 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:48:17.570726 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:48:17.570726 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:48:17.570726 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:48:17.570726 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:48:17.570726 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:48:17.570726 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:48:17.570726 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:48:17.570726 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:48:17.570726 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:48:17.570726 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:48:17.570726 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 10 12:48:17.570726 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 10 12:48:17.570726 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 10 12:48:17.571810 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 10 12:48:17.571810 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:48:17.571810 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:48:17.571810 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:48:17.571810 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:48:17.571810 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:48:17.571810 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:48:17.571810 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:48:17.571810 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:48:17.571810 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:48:17.571810 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:48:17.571810 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:48:17.571810 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:48:17.571810 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:48:17.571810 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:48:17.571810 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:48:17.571810 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:48:17.572979 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 10 12:48:17.572979 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 10 12:48:17.572979 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:48:17.572979 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:48:17.572979 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 622/1237] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:48:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:48:17.577853 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d105f6d2-2549-4f72-b1c2-9447dac2c956 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:48:17.677495 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d105f6d2-2549-4f72-b1c2-9447dac2c956 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:48:17.686002 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:48:17.710526 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeac5db0-0680-4c6c-9747-96eb2bed2abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:48:17.711217 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeac5db0-0680-4c6c-9747-96eb2bed2abb None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:17.711472 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeac5db0-0680-4c6c-9747-96eb2bed2abb None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:17.711751 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeac5db0-0680-4c6c-9747-96eb2bed2abb None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:17.712514 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeac5db0-0680-4c6c-9747-96eb2bed2abb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:48:17.712820 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeac5db0-0680-4c6c-9747-96eb2bed2abb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:48:17.714182 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeac5db0-0680-4c6c-9747-96eb2bed2abb None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:48:17.725629 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 616/1238] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Jun 10 12:48:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:48:17.731257 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f56787-a9e5-49bd-827d-efc550b76765 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:48:17.829659 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f56787-a9e5-49bd-827d-efc550b76765 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:48:17.863391 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9bd1a4-c284-428c-a685-021476fe8a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:48:17.864222 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9bd1a4-c284-428c-a685-021476fe8a44 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:17.864510 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9bd1a4-c284-428c-a685-021476fe8a44 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:17.864854 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9bd1a4-c284-428c-a685-021476fe8a44 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:17.867548 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:48:17.872641 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9bd1a4-c284-428c-a685-021476fe8a44 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:48:17.874662 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9bd1a4-c284-428c-a685-021476fe8a44 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:48:17.888529 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-bf9bd1a4-c284-428c-a685-021476fe8a44 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 10 12:48:17.888529 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:48:17.888529 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:48:17.888529 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:48:17.888529 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:48:17.888529 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:48:17.888529 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:48:17.888529 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:48:17.888529 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:48:17.888529 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:48:17.888529 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:48:17.888529 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:48:17.888529 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 10 12:48:17.888529 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 10 12:48:17.888529 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 10 12:48:17.888529 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 10 12:48:17.889744 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:48:17.889744 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:48:17.889744 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 10 12:48:17.889744 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 10 12:48:17.889744 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 10 12:48:17.889744 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 10 12:48:17.889744 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:48:17.889744 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:48:17.889744 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:48:17.889744 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:48:17.889744 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:48:17.889744 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:48:17.889744 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:48:17.889744 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:48:17.889744 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:48:17.889744 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:48:17.889744 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:48:17.889744 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:48:17.890783 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:48:17.890783 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:48:17.890783 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 10 12:48:17.890783 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 10 12:48:17.890783 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 10 12:48:17.890783 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 10 12:48:17.890783 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 10 12:48:17.890783 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 10 12:48:17.890783 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 10 12:48:17.890783 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:48:17.890783 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 623/1239] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:48:17 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:48:17.895458 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeac5db0-0680-4c6c-9747-96eb2bed2abb None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:48:17.996878 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeac5db0-0680-4c6c-9747-96eb2bed2abb None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:48:18.029466 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53639559-2e8e-44a0-ad95-4eea6ac7bac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:48:18.030637 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53639559-2e8e-44a0-ad95-4eea6ac7bac3 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:18.031184 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53639559-2e8e-44a0-ad95-4eea6ac7bac3 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:18.031453 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53639559-2e8e-44a0-ad95-4eea6ac7bac3 None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:18.032325 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53639559-2e8e-44a0-ad95-4eea6ac7bac3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:48:18.032633 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53639559-2e8e-44a0-ad95-4eea6ac7bac3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:48:18.034058 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53639559-2e8e-44a0-ad95-4eea6ac7bac3 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:48:18.051032 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 617/1240] 10.222.0.27 () {56 vars in 1143 bytes} [Fri Jun 10 12:48:17 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:48:18.056829 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9bd1a4-c284-428c-a685-021476fe8a44 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:48:18.157208 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9bd1a4-c284-428c-a685-021476fe8a44 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:48:18.189978 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff434e87-5274-4555-a2e5-21e70deeb27f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:48:18.190683 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff434e87-5274-4555-a2e5-21e70deeb27f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:18.190976 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff434e87-5274-4555-a2e5-21e70deeb27f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:18.191259 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff434e87-5274-4555-a2e5-21e70deeb27f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:18.250512 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:48:18.256767 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff434e87-5274-4555-a2e5-21e70deeb27f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:48:18.258206 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff434e87-5274-4555-a2e5-21e70deeb27f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:48:18.354491 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 624/1241] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Jun 10 12:48:18 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 10 12:48:18.360440 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53639559-2e8e-44a0-ad95-4eea6ac7bac3 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:48:18.387032 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:48:18.458074 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53639559-2e8e-44a0-ad95-4eea6ac7bac3 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:48:18.491062 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aa0581-b8a9-4909-91da-c33ea1067c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:48:18.491845 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aa0581-b8a9-4909-91da-c33ea1067c9a None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:18.492162 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aa0581-b8a9-4909-91da-c33ea1067c9a None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:18.492443 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aa0581-b8a9-4909-91da-c33ea1067c9a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:18.502633 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38aa0581-b8a9-4909-91da-c33ea1067c9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:48:18.504108 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38aa0581-b8a9-4909-91da-c33ea1067c9a None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:48:18.522694 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-38aa0581-b8a9-4909-91da-c33ea1067c9a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:48:18.522694 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:48:18.522694 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:48:18.522694 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:48:18.522694 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:48:18.522694 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:48:18.522694 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:48:18.522694 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:48:18.522694 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:48:18.522694 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:48:18.522694 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:48:18.522694 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:48:18.522694 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 10 12:48:18.522694 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 10 12:48:18.522694 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 10 12:48:18.523759 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 10 12:48:18.523759 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:48:18.523759 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:48:18.523759 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:48:18.523759 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:48:18.523759 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:48:18.523759 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:48:18.523759 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:48:18.523759 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:48:18.523759 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:48:18.523759 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:48:18.523759 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:48:18.523759 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:48:18.523759 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:48:18.523759 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:48:18.523759 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:48:18.523759 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:48:18.525069 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 10 12:48:18.525069 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 10 12:48:18.525069 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:48:18.525069 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:48:18.525069 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 618/1242] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:48:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:48:18.529572 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff434e87-5274-4555-a2e5-21e70deeb27f None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:48:18.621268 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:48:18.627417 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff434e87-5274-4555-a2e5-21e70deeb27f None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:48:18.660540 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da02aa3-a641-473d-a155-10e8dd59c302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:48:18.661194 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da02aa3-a641-473d-a155-10e8dd59c302 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:18.661483 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da02aa3-a641-473d-a155-10e8dd59c302 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:18.661764 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da02aa3-a641-473d-a155-10e8dd59c302 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:18.662484 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da02aa3-a641-473d-a155-10e8dd59c302 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:48:18.662792 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da02aa3-a641-473d-a155-10e8dd59c302 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:48:18.664180 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da02aa3-a641-473d-a155-10e8dd59c302 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:48:18.675001 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 625/1243] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Jun 10 12:48:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:48:18.680593 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aa0581-b8a9-4909-91da-c33ea1067c9a None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:48:18.756926 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:48:18.778321 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aa0581-b8a9-4909-91da-c33ea1067c9a None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:48:18.811211 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fe833a-2953-40d2-bbba-4c715914b07a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:48:18.812611 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fe833a-2953-40d2-bbba-4c715914b07a None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:18.812893 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fe833a-2953-40d2-bbba-4c715914b07a None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:18.813162 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fe833a-2953-40d2-bbba-4c715914b07a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:18.820484 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fe833a-2953-40d2-bbba-4c715914b07a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:48:18.821992 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fe833a-2953-40d2-bbba-4c715914b07a None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:48:18.835986 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-01fe833a-2953-40d2-bbba-4c715914b07a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 10 12:48:18.835986 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:48:18.835986 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:48:18.835986 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:48:18.835986 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:48:18.835986 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:48:18.835986 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:48:18.835986 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:48:18.835986 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:48:18.835986 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:48:18.835986 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:48:18.835986 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:48:18.835986 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 10 12:48:18.835986 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 10 12:48:18.835986 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 10 12:48:18.837111 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 10 12:48:18.837111 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:48:18.837111 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:48:18.837111 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:48:18.837111 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:48:18.837111 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:48:18.837111 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:48:18.837111 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:48:18.837111 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:48:18.837111 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:48:18.837111 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:48:18.837111 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:48:18.837111 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:48:18.837111 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:48:18.837111 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:48:18.837111 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:48:18.837111 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:48:18.838293 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 10 12:48:18.838293 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 10 12:48:18.838293 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 10 12:48:18.838293 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 10 12:48:18.838293 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 10 12:48:18.838293 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 10 12:48:18.838293 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 10 12:48:18.838293 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:48:18.838293 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 619/1244] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Jun 10 12:48:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:48:18.842899 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da02aa3-a641-473d-a155-10e8dd59c302 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:48:18.940736 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da02aa3-a641-473d-a155-10e8dd59c302 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:48:18.973005 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a4ee72-2179-4cce-b6fa-20504a0f7aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:48:18.973653 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a4ee72-2179-4cce-b6fa-20504a0f7aa1 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:18.973934 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a4ee72-2179-4cce-b6fa-20504a0f7aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:18.974208 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a4ee72-2179-4cce-b6fa-20504a0f7aa1 None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:18.974988 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a4ee72-2179-4cce-b6fa-20504a0f7aa1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:48:18.975293 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a4ee72-2179-4cce-b6fa-20504a0f7aa1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:48:18.976733 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a4ee72-2179-4cce-b6fa-20504a0f7aa1 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:48:18.989052 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 626/1245] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Jun 10 12:48:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:48:18.995001 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fe833a-2953-40d2-bbba-4c715914b07a None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:48:19.092317 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fe833a-2953-40d2-bbba-4c715914b07a None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:48:19.125871 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0117d192-2d46-4446-ac64-167cd083e396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:48:19.126511 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0117d192-2d46-4446-ac64-167cd083e396 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:19.126794 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0117d192-2d46-4446-ac64-167cd083e396 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:19.127070 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0117d192-2d46-4446-ac64-167cd083e396 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:19.127937 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0117d192-2d46-4446-ac64-167cd083e396 None admin] RBAC: Adding query filter params (role.id=ea7e8f74523b4322857fb3f4307d171e, scope.project.id=73ea5b23a2d34544ab493f4c5e4034b8, user.id=d972ab782b3f4d2aba575314f16dafc7) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:48:19.128327 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0117d192-2d46-4446-ac64-167cd083e396 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:48:19.129829 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0117d192-2d46-4446-ac64-167cd083e396 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:48:19.135521 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 620/1246] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Jun 10 12:48:18 2022] GET /identity/v3/role_assignments?user.id=d972ab782b3f4d2aba575314f16dafc7&scope.project.id=73ea5b23a2d34544ab493f4c5e4034b8&role.id=ea7e8f74523b4322857fb3f4307d171e => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:48:21.263084 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9768bca7-80c7-4dd0-80c4-495ae9b7bb9b None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:21.263606 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9768bca7-80c7-4dd0-80c4-495ae9b7bb9b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:21.263606 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9768bca7-80c7-4dd0-80c4-495ae9b7bb9b None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:21.264965 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 627/1247] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:48:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:48:21.270043 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a4b54b-04a9-455a-af23-731e7cf8b4f8 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:21.277070 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a4b54b-04a9-455a-af23-731e7cf8b4f8 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:21.277070 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a4b54b-04a9-455a-af23-731e7cf8b4f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:21.277070 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:48:21.287872 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-77a4b54b-04a9-455a-af23-731e7cf8b4f8 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:48:21.289159 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-77a4b54b-04a9-455a-af23-731e7cf8b4f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:48:21.354395 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 621/1248] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:48:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 10 12:48:21.378567 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9768bca7-80c7-4dd0-80c4-495ae9b7bb9b None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:48:21.397495 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:48:21.482043 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9768bca7-80c7-4dd0-80c4-495ae9b7bb9b None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:48:21.515934 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fbf6f0-dfe4-4516-9c59-214aa84207fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:48:21.516666 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fbf6f0-dfe4-4516-9c59-214aa84207fa None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:21.516953 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fbf6f0-dfe4-4516-9c59-214aa84207fa None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:21.517237 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fbf6f0-dfe4-4516-9c59-214aa84207fa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:21.527136 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fbf6f0-dfe4-4516-9c59-214aa84207fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:48:21.535372 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-57fbf6f0-dfe4-4516-9c59-214aa84207fa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 10 12:48:21.535372 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:48:21.535372 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:48:21.535372 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:48:21.535372 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:48:21.535372 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:48:21.535372 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:48:21.535372 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:48:21.535372 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:48:21.535372 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:48:21.535372 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:48:21.535372 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:48:21.535372 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:48:21.535372 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:48:21.535372 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:48:21.535372 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:48:21.537210 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:48:21.537210 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:48:21.537210 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:48:21.537210 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:48:21.537210 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:48:21.537210 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:48:21.537210 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:48:21.537210 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:48:21.537210 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:48:21.537210 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:48:21.537210 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:48:21.537210 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:48:21.537210 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:48:21.537210 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:48:21.537210 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:48:21.537210 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:48:21.538143 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:48:21.538143 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:48:21.538143 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:48:21.538143 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:48:21.538143 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:48:21.538143 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:48:21.538143 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:48:21.538143 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:48:21.538143 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:48:21.538143 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:48:21.538143 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 10 12:48:21.538143 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:48:21.538143 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 628/1249] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Jun 10 12:48:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:48:21.542764 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a4b54b-04a9-455a-af23-731e7cf8b4f8 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:48:21.644875 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a4b54b-04a9-455a-af23-731e7cf8b4f8 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:48:21.675042 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:48:21.679817 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2864769a-88bd-4e45-80bf-77e7b424d96b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:48:21.680508 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2864769a-88bd-4e45-80bf-77e7b424d96b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:21.680788 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2864769a-88bd-4e45-80bf-77e7b424d96b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:21.681062 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2864769a-88bd-4e45-80bf-77e7b424d96b None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:21.681797 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2864769a-88bd-4e45-80bf-77e7b424d96b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:48:21.682099 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2864769a-88bd-4e45-80bf-77e7b424d96b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:48:21.683506 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2864769a-88bd-4e45-80bf-77e7b424d96b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:48:21.691319 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 622/1250] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Jun 10 12:48:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:48:21.697094 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fbf6f0-dfe4-4516-9c59-214aa84207fa None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:48:21.773713 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:48:21.796763 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fbf6f0-dfe4-4516-9c59-214aa84207fa None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:48:21.828927 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256468ea-363a-4434-8bfa-3bf7cfacbc15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:48:21.829705 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256468ea-363a-4434-8bfa-3bf7cfacbc15 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:21.829986 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256468ea-363a-4434-8bfa-3bf7cfacbc15 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:21.830315 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256468ea-363a-4434-8bfa-3bf7cfacbc15 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:21.840734 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256468ea-363a-4434-8bfa-3bf7cfacbc15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:48:21.842165 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256468ea-363a-4434-8bfa-3bf7cfacbc15 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:48:21.859955 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-256468ea-363a-4434-8bfa-3bf7cfacbc15 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:48:21.859955 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:48:21.859955 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:48:21.859955 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:48:21.859955 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:48:21.859955 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:48:21.859955 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:48:21.859955 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:48:21.859955 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:48:21.859955 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:48:21.859955 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:48:21.859955 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:48:21.859955 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 10 12:48:21.859955 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 10 12:48:21.859955 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 10 12:48:21.860944 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 10 12:48:21.860944 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:48:21.860944 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:48:21.860944 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:48:21.860944 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:48:21.860944 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:48:21.860944 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:48:21.860944 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:48:21.860944 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:48:21.860944 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:48:21.860944 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:48:21.860944 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:48:21.860944 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:48:21.860944 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:48:21.860944 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:48:21.860944 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:48:21.860944 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:48:21.861864 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 10 12:48:21.861864 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 10 12:48:21.861864 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:48:21.861864 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:48:21.861864 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 629/1251] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:48:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:48:21.866829 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2864769a-88bd-4e45-80bf-77e7b424d96b None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:48:21.964285 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2864769a-88bd-4e45-80bf-77e7b424d96b None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:48:21.998149 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c6679e-2227-47b2-aa2a-efcf3b19b882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:48:22.001639 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c6679e-2227-47b2-aa2a-efcf3b19b882 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:22.001639 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c6679e-2227-47b2-aa2a-efcf3b19b882 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:22.001639 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c6679e-2227-47b2-aa2a-efcf3b19b882 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:22.001639 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c6679e-2227-47b2-aa2a-efcf3b19b882 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:48:22.001639 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c6679e-2227-47b2-aa2a-efcf3b19b882 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:48:22.002063 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c6679e-2227-47b2-aa2a-efcf3b19b882 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:48:22.013115 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 623/1252] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Jun 10 12:48:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:48:22.019256 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256468ea-363a-4434-8bfa-3bf7cfacbc15 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:48:22.118335 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256468ea-363a-4434-8bfa-3bf7cfacbc15 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:48:22.143671 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:48:22.151150 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d85192-972d-4514-8dea-a6f74166277b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:48:22.151978 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d85192-972d-4514-8dea-a6f74166277b None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:22.152268 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d85192-972d-4514-8dea-a6f74166277b None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:22.152542 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d85192-972d-4514-8dea-a6f74166277b None admin] PATH_INFO: `/v3/users/placement` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:22.160544 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d85192-972d-4514-8dea-a6f74166277b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:48:22.162367 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d85192-972d-4514-8dea-a6f74166277b None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:48:22.175912 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-c0d85192-972d-4514-8dea-a6f74166277b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 10 12:48:22.175912 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:48:22.175912 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:48:22.175912 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:48:22.175912 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:48:22.175912 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:48:22.175912 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:48:22.175912 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:48:22.175912 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:48:22.175912 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:48:22.175912 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:48:22.175912 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:48:22.175912 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 10 12:48:22.175912 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 10 12:48:22.175912 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 10 12:48:22.175912 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 10 12:48:22.177036 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:48:22.177036 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:48:22.177036 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 10 12:48:22.177036 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 10 12:48:22.177036 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 10 12:48:22.177036 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 10 12:48:22.177036 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:48:22.177036 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:48:22.177036 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:48:22.177036 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:48:22.177036 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:48:22.177036 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:48:22.177036 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:48:22.177036 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:48:22.177036 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:48:22.177036 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:48:22.177036 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:48:22.177036 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:48:22.178214 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:48:22.178214 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:48:22.178214 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 10 12:48:22.178214 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 10 12:48:22.178214 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 10 12:48:22.178214 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 10 12:48:22.178214 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 10 12:48:22.178214 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 10 12:48:22.178214 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 10 12:48:22.178214 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:48:22.178214 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 630/1253] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:48:22 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:48:22.183025 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c6679e-2227-47b2-aa2a-efcf3b19b882 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:48:22.237386 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:48:22.281581 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c6679e-2227-47b2-aa2a-efcf3b19b882 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:48:22.314669 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417208df-4d4f-4419-8c37-b7499aeaf2c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:48:22.315331 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417208df-4d4f-4419-8c37-b7499aeaf2c1 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:22.315620 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417208df-4d4f-4419-8c37-b7499aeaf2c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:22.315925 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417208df-4d4f-4419-8c37-b7499aeaf2c1 None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:22.316757 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417208df-4d4f-4419-8c37-b7499aeaf2c1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:48:22.317064 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417208df-4d4f-4419-8c37-b7499aeaf2c1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:48:22.318479 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417208df-4d4f-4419-8c37-b7499aeaf2c1 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:48:22.335359 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 624/1254] 10.222.0.27 () {56 vars in 1143 bytes} [Fri Jun 10 12:48:22 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:48:22.343628 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d85192-972d-4514-8dea-a6f74166277b None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:48:22.447673 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d85192-972d-4514-8dea-a6f74166277b None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:48:22.480940 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fddba8-1ca7-4fe2-8c05-b068e20a751e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:48:22.481648 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fddba8-1ca7-4fe2-8c05-b068e20a751e None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:22.481937 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fddba8-1ca7-4fe2-8c05-b068e20a751e None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:22.482241 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fddba8-1ca7-4fe2-8c05-b068e20a751e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:22.536708 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:48:22.548386 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fddba8-1ca7-4fe2-8c05-b068e20a751e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:48:22.549833 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fddba8-1ca7-4fe2-8c05-b068e20a751e None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:48:22.644490 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 631/1255] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Jun 10 12:48:22 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 10 12:48:22.650723 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417208df-4d4f-4419-8c37-b7499aeaf2c1 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:48:22.749952 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417208df-4d4f-4419-8c37-b7499aeaf2c1 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:48:22.763024 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:48:22.783390 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c532ed0-d3a0-4965-9fd0-0e114ba7ae78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:48:22.784207 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c532ed0-d3a0-4965-9fd0-0e114ba7ae78 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:22.784488 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c532ed0-d3a0-4965-9fd0-0e114ba7ae78 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:22.784761 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c532ed0-d3a0-4965-9fd0-0e114ba7ae78 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:22.794800 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c532ed0-d3a0-4965-9fd0-0e114ba7ae78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:48:22.796840 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c532ed0-d3a0-4965-9fd0-0e114ba7ae78 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:48:22.814275 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-4c532ed0-d3a0-4965-9fd0-0e114ba7ae78 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:48:22.814275 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:48:22.814275 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:48:22.814275 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:48:22.814275 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:48:22.814275 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:48:22.814275 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:48:22.814275 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:48:22.814275 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:48:22.814275 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:48:22.814275 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:48:22.814275 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:48:22.814275 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 10 12:48:22.814275 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 10 12:48:22.814275 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 10 12:48:22.820519 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 10 12:48:22.820519 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:48:22.820519 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:48:22.820519 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:48:22.820519 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:48:22.820519 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:48:22.820519 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:48:22.820519 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:48:22.820519 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:48:22.820519 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:48:22.820519 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:48:22.820519 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:48:22.820519 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:48:22.820519 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:48:22.820519 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:48:22.820519 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:48:22.820519 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:48:22.821621 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 10 12:48:22.821621 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 10 12:48:22.821621 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:48:22.821621 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:48:22.821621 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 625/1256] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:48:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:48:22.821983 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fddba8-1ca7-4fe2-8c05-b068e20a751e None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:48:22.907122 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:48:22.918759 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fddba8-1ca7-4fe2-8c05-b068e20a751e None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:48:22.952430 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7f8782-ae5c-406b-8bdb-aa60679f33a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:48:22.953081 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7f8782-ae5c-406b-8bdb-aa60679f33a2 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:22.953363 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7f8782-ae5c-406b-8bdb-aa60679f33a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:22.953636 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7f8782-ae5c-406b-8bdb-aa60679f33a2 None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:22.954376 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7f8782-ae5c-406b-8bdb-aa60679f33a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:48:22.954675 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7f8782-ae5c-406b-8bdb-aa60679f33a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:48:22.956036 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7f8782-ae5c-406b-8bdb-aa60679f33a2 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:48:22.966942 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 632/1257] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Jun 10 12:48:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:48:22.972670 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c532ed0-d3a0-4965-9fd0-0e114ba7ae78 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:48:23.076480 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c532ed0-d3a0-4965-9fd0-0e114ba7ae78 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:48:23.109994 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19932783-345c-4a69-9956-1ceb0eccc189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:48:23.110775 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19932783-345c-4a69-9956-1ceb0eccc189 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:23.111058 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19932783-345c-4a69-9956-1ceb0eccc189 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:23.111334 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19932783-345c-4a69-9956-1ceb0eccc189 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:23.117896 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19932783-345c-4a69-9956-1ceb0eccc189 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:48:23.120403 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19932783-345c-4a69-9956-1ceb0eccc189 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:48:23.132598 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-19932783-345c-4a69-9956-1ceb0eccc189 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 10 12:48:23.132598 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:48:23.132598 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:48:23.132598 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:48:23.132598 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:48:23.132598 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:48:23.132598 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:48:23.132598 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:48:23.132598 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:48:23.132598 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:48:23.132598 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:48:23.132598 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:48:23.132598 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 10 12:48:23.132598 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 10 12:48:23.132598 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 10 12:48:23.133637 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 10 12:48:23.133637 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:48:23.133637 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:48:23.133637 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:48:23.133637 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:48:23.133637 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:48:23.133637 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:48:23.133637 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:48:23.133637 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:48:23.133637 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:48:23.133637 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:48:23.133637 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:48:23.133637 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:48:23.133637 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:48:23.133637 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:48:23.133637 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:48:23.133637 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:48:23.134709 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 10 12:48:23.134709 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 10 12:48:23.134709 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 10 12:48:23.134709 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 10 12:48:23.134709 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 10 12:48:23.134709 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 10 12:48:23.134709 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 10 12:48:23.134709 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:48:23.134709 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 626/1258] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Jun 10 12:48:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:48:23.139635 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7f8782-ae5c-406b-8bdb-aa60679f33a2 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:48:23.239962 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7f8782-ae5c-406b-8bdb-aa60679f33a2 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:48:23.272113 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152e7565-2f49-4dfc-865e-4ddf7b51c6a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:48:23.272767 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152e7565-2f49-4dfc-865e-4ddf7b51c6a4 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:23.273051 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152e7565-2f49-4dfc-865e-4ddf7b51c6a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:23.273325 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152e7565-2f49-4dfc-865e-4ddf7b51c6a4 None admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:23.274106 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152e7565-2f49-4dfc-865e-4ddf7b51c6a4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:48:23.274415 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152e7565-2f49-4dfc-865e-4ddf7b51c6a4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:48:23.275844 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152e7565-2f49-4dfc-865e-4ddf7b51c6a4 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:48:23.288926 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 633/1259] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Jun 10 12:48:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:48:23.294960 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19932783-345c-4a69-9956-1ceb0eccc189 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:48:23.321915 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:48:23.394418 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19932783-345c-4a69-9956-1ceb0eccc189 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:48:23.428594 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9dfdcf-96bf-40b1-a63c-c3bbe20be683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:48:23.429240 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9dfdcf-96bf-40b1-a63c-c3bbe20be683 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:23.429520 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9dfdcf-96bf-40b1-a63c-c3bbe20be683 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:23.429795 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9dfdcf-96bf-40b1-a63c-c3bbe20be683 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:23.430620 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9dfdcf-96bf-40b1-a63c-c3bbe20be683 None admin] RBAC: Adding query filter params (role.id=b43d5a8247e244869c2c70f986f42e8a, scope.project.id=73ea5b23a2d34544ab493f4c5e4034b8, user.id=d972ab782b3f4d2aba575314f16dafc7) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:48:23.430930 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9dfdcf-96bf-40b1-a63c-c3bbe20be683 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:48:23.432383 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9dfdcf-96bf-40b1-a63c-c3bbe20be683 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:48:23.437781 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 627/1260] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Jun 10 12:48:23 2022] GET /identity/v3/role_assignments?user.id=d972ab782b3f4d2aba575314f16dafc7&scope.project.id=73ea5b23a2d34544ab493f4c5e4034b8&role.id=b43d5a8247e244869c2c70f986f42e8a => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:48:25.494470 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e17292-a344-4518-9c4d-38e07728c638 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:25.495031 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e17292-a344-4518-9c4d-38e07728c638 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:25.495031 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e17292-a344-4518-9c4d-38e07728c638 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:25.496408 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 634/1261] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:48:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:48:25.502579 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047c9e1c-ee61-4021-9e7d-578d8f531149 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:25.508716 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047c9e1c-ee61-4021-9e7d-578d8f531149 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:25.508716 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047c9e1c-ee61-4021-9e7d-578d8f531149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:25.520679 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-047c9e1c-ee61-4021-9e7d-578d8f531149 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:48:25.521872 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-047c9e1c-ee61-4021-9e7d-578d8f531149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:48:25.586856 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 628/1262] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:48:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 10 12:48:25.609371 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e17292-a344-4518-9c4d-38e07728c638 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:48:25.620140 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:48:25.708120 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e17292-a344-4518-9c4d-38e07728c638 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:48:25.741143 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321963ea-8bbd-4b39-8810-d7fbf2ff70d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:48:25.741922 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321963ea-8bbd-4b39-8810-d7fbf2ff70d0 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:25.742203 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321963ea-8bbd-4b39-8810-d7fbf2ff70d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:25.742475 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321963ea-8bbd-4b39-8810-d7fbf2ff70d0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:25.753216 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321963ea-8bbd-4b39-8810-d7fbf2ff70d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:48:25.760802 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-321963ea-8bbd-4b39-8810-d7fbf2ff70d0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 10 12:48:25.760802 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:48:25.760802 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:48:25.760802 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:48:25.760802 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:48:25.760802 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:48:25.760802 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:48:25.760802 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:48:25.760802 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:48:25.760802 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:48:25.760802 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:48:25.760802 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:48:25.760802 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:48:25.760802 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:48:25.760802 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:48:25.760802 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:48:25.761958 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:48:25.761958 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:48:25.761958 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:48:25.761958 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:48:25.761958 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:48:25.761958 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:48:25.761958 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:48:25.761958 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:48:25.761958 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:48:25.761958 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:48:25.761958 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:48:25.761958 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:48:25.761958 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:48:25.761958 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:48:25.761958 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:48:25.761958 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:48:25.762895 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:48:25.762895 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:48:25.762895 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:48:25.762895 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:48:25.762895 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:48:25.762895 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:48:25.762895 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:48:25.762895 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:48:25.762895 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:48:25.762895 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:48:25.762895 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 10 12:48:25.762895 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:48:25.762895 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 635/1263] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Jun 10 12:48:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:48:25.768014 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047c9e1c-ee61-4021-9e7d-578d8f531149 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:48:25.785930 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:48:25.863030 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047c9e1c-ee61-4021-9e7d-578d8f531149 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:48:25.902577 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c01e32-be0f-4dba-8ee5-c057a8928f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:48:25.903238 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c01e32-be0f-4dba-8ee5-c057a8928f58 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:25.903519 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c01e32-be0f-4dba-8ee5-c057a8928f58 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:25.903793 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c01e32-be0f-4dba-8ee5-c057a8928f58 None admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:25.904557 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c01e32-be0f-4dba-8ee5-c057a8928f58 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:48:25.904862 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c01e32-be0f-4dba-8ee5-c057a8928f58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:48:25.906875 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c01e32-be0f-4dba-8ee5-c057a8928f58 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:48:25.914699 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 629/1264] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Jun 10 12:48:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:48:25.920495 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321963ea-8bbd-4b39-8810-d7fbf2ff70d0 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:48:25.984932 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:48:26.017176 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321963ea-8bbd-4b39-8810-d7fbf2ff70d0 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:48:26.049823 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4d6cf5-39c5-481f-a73b-0934eedd5ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:48:26.050654 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4d6cf5-39c5-481f-a73b-0934eedd5ee6 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:26.051169 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4d6cf5-39c5-481f-a73b-0934eedd5ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:26.051511 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4d6cf5-39c5-481f-a73b-0934eedd5ee6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:26.062640 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4d6cf5-39c5-481f-a73b-0934eedd5ee6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:48:26.064108 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4d6cf5-39c5-481f-a73b-0934eedd5ee6 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:48:26.081328 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-2e4d6cf5-39c5-481f-a73b-0934eedd5ee6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:48:26.081328 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:48:26.081328 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:48:26.081328 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:48:26.081328 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:48:26.081328 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:48:26.081328 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:48:26.081328 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:48:26.081328 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:48:26.081328 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:48:26.081328 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:48:26.081328 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:48:26.081328 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 10 12:48:26.081328 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 10 12:48:26.081328 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 10 12:48:26.082403 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 10 12:48:26.082403 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:48:26.082403 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:48:26.082403 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:48:26.082403 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:48:26.082403 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:48:26.082403 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:48:26.082403 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:48:26.082403 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:48:26.082403 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:48:26.082403 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:48:26.082403 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:48:26.082403 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:48:26.082403 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:48:26.082403 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:48:26.082403 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:48:26.082403 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:48:26.083535 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 10 12:48:26.083535 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 10 12:48:26.083535 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:48:26.083535 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:48:26.083535 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 636/1265] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:48:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:48:26.088256 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c01e32-be0f-4dba-8ee5-c057a8928f58 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:48:26.166646 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:48:26.184881 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c01e32-be0f-4dba-8ee5-c057a8928f58 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:48:26.217760 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087b23a6-4f01-431e-9118-83743e1e95aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:48:26.218404 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087b23a6-4f01-431e-9118-83743e1e95aa None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:26.218684 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087b23a6-4f01-431e-9118-83743e1e95aa None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:26.218982 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087b23a6-4f01-431e-9118-83743e1e95aa None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:26.219670 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087b23a6-4f01-431e-9118-83743e1e95aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:48:26.220005 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087b23a6-4f01-431e-9118-83743e1e95aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:48:26.221362 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087b23a6-4f01-431e-9118-83743e1e95aa None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:48:26.232188 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 630/1266] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Jun 10 12:48:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:48:26.237893 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4d6cf5-39c5-481f-a73b-0934eedd5ee6 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:48:26.338339 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4d6cf5-39c5-481f-a73b-0934eedd5ee6 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:48:26.357723 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:48:26.377931 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd69008c-0c60-46ec-8a51-c09df6668eae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:48:26.379115 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd69008c-0c60-46ec-8a51-c09df6668eae None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:26.379645 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd69008c-0c60-46ec-8a51-c09df6668eae None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:26.380110 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd69008c-0c60-46ec-8a51-c09df6668eae None admin] PATH_INFO: `/v3/users/placement` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:26.388638 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd69008c-0c60-46ec-8a51-c09df6668eae None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:48:26.390754 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd69008c-0c60-46ec-8a51-c09df6668eae None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:48:26.404985 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-dd69008c-0c60-46ec-8a51-c09df6668eae None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 10 12:48:26.404985 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:48:26.404985 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:48:26.404985 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:48:26.404985 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:48:26.404985 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:48:26.404985 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:48:26.404985 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:48:26.404985 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:48:26.404985 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:48:26.404985 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:48:26.404985 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:48:26.404985 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 10 12:48:26.404985 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 10 12:48:26.404985 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 10 12:48:26.404985 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 10 12:48:26.406395 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:48:26.406395 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:48:26.406395 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 10 12:48:26.406395 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 10 12:48:26.406395 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 10 12:48:26.406395 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 10 12:48:26.406395 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:48:26.406395 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:48:26.406395 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:48:26.406395 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:48:26.406395 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:48:26.406395 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:48:26.406395 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:48:26.406395 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:48:26.406395 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:48:26.406395 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:48:26.406395 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:48:26.406395 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:48:26.407936 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:48:26.407936 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:48:26.407936 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 10 12:48:26.407936 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 10 12:48:26.407936 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 10 12:48:26.407936 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 10 12:48:26.407936 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 10 12:48:26.407936 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 10 12:48:26.407936 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 10 12:48:26.407936 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:48:26.407936 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 637/1267] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:48:26 2022] GET /identity/v3/users/placement => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:48:26.412533 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087b23a6-4f01-431e-9118-83743e1e95aa None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:48:26.511788 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087b23a6-4f01-431e-9118-83743e1e95aa None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:48:26.544335 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f066c01-cbc8-477e-9d9b-d984b57f5c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:48:26.544996 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f066c01-cbc8-477e-9d9b-d984b57f5c55 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:26.545281 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f066c01-cbc8-477e-9d9b-d984b57f5c55 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:26.545557 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f066c01-cbc8-477e-9d9b-d984b57f5c55 None admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:26.546924 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f066c01-cbc8-477e-9d9b-d984b57f5c55 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:48:26.547229 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f066c01-cbc8-477e-9d9b-d984b57f5c55 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:48:26.548687 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f066c01-cbc8-477e-9d9b-d984b57f5c55 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:48:26.557365 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:48:26.570097 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 631/1268] 10.222.0.27 () {56 vars in 1143 bytes} [Fri Jun 10 12:48:26 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 12:48:26.575702 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd69008c-0c60-46ec-8a51-c09df6668eae None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:48:26.672716 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd69008c-0c60-46ec-8a51-c09df6668eae None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:48:26.710222 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f3f967-5020-4334-ad8f-119f8e18e9f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:48:26.711075 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f3f967-5020-4334-ad8f-119f8e18e9f4 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:26.711355 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f3f967-5020-4334-ad8f-119f8e18e9f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:26.711628 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f3f967-5020-4334-ad8f-119f8e18e9f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:26.723248 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f3f967-5020-4334-ad8f-119f8e18e9f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:48:26.724729 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f3f967-5020-4334-ad8f-119f8e18e9f4 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:48:26.742295 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-26f3f967-5020-4334-ad8f-119f8e18e9f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:48:26.742295 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:48:26.742295 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:48:26.742295 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:48:26.742295 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:48:26.742295 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:48:26.742295 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:48:26.742295 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:48:26.742295 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:48:26.742295 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:48:26.742295 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:48:26.742295 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:48:26.742295 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 10 12:48:26.742295 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 10 12:48:26.742295 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 10 12:48:26.743353 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 10 12:48:26.743353 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:48:26.743353 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:48:26.743353 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:48:26.743353 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:48:26.743353 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:48:26.743353 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:48:26.743353 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:48:26.743353 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:48:26.743353 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:48:26.743353 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:48:26.743353 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:48:26.743353 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:48:26.743353 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:48:26.743353 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:48:26.743353 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:48:26.743353 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:48:26.744320 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 10 12:48:26.744320 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 10 12:48:26.744320 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:48:26.744320 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:48:26.744320 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 638/1269] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:48:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:48:26.749949 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f066c01-cbc8-477e-9d9b-d984b57f5c55 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:48:26.847518 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f066c01-cbc8-477e-9d9b-d984b57f5c55 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:48:26.883023 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaba2d03-20a8-4820-9774-fdc5af700418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:48:26.883684 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaba2d03-20a8-4820-9774-fdc5af700418 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:26.884006 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaba2d03-20a8-4820-9774-fdc5af700418 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:26.884287 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaba2d03-20a8-4820-9774-fdc5af700418 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:26.885005 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaba2d03-20a8-4820-9774-fdc5af700418 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:48:26.885310 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaba2d03-20a8-4820-9774-fdc5af700418 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:48:26.886668 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaba2d03-20a8-4820-9774-fdc5af700418 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:48:26.900387 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 632/1270] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Jun 10 12:48:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:48:26.903612 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f3f967-5020-4334-ad8f-119f8e18e9f4 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:48:26.937198 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:48:27.006497 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f3f967-5020-4334-ad8f-119f8e18e9f4 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:48:27.040900 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575a2647-2869-4ba2-b541-f1fe1365b1da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:48:27.041693 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575a2647-2869-4ba2-b541-f1fe1365b1da None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:27.041979 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575a2647-2869-4ba2-b541-f1fe1365b1da None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:27.042282 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575a2647-2869-4ba2-b541-f1fe1365b1da None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:27.049664 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575a2647-2869-4ba2-b541-f1fe1365b1da None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:48:27.051097 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575a2647-2869-4ba2-b541-f1fe1365b1da None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:48:27.065300 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-575a2647-2869-4ba2-b541-f1fe1365b1da None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 10 12:48:27.065300 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:48:27.065300 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:48:27.065300 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:48:27.065300 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:48:27.065300 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:48:27.065300 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:48:27.065300 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:48:27.065300 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:48:27.065300 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:48:27.065300 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:48:27.065300 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:48:27.065300 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 10 12:48:27.065300 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 10 12:48:27.065300 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 10 12:48:27.066396 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 10 12:48:27.066396 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:48:27.066396 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:48:27.066396 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:48:27.066396 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:48:27.066396 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:48:27.066396 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:48:27.066396 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:48:27.066396 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:48:27.066396 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:48:27.066396 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:48:27.066396 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:48:27.066396 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:48:27.066396 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:48:27.066396 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:48:27.066396 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:48:27.066396 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:48:27.067419 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 10 12:48:27.067419 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 10 12:48:27.067419 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 10 12:48:27.067419 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 10 12:48:27.067419 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 10 12:48:27.067419 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 10 12:48:27.067419 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 10 12:48:27.067419 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:48:27.067419 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 639/1271] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Jun 10 12:48:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 10 12:48:27.072294 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaba2d03-20a8-4820-9774-fdc5af700418 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:48:27.130410 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:48:27.171556 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaba2d03-20a8-4820-9774-fdc5af700418 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:48:27.202941 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbebcc7-e1b3-43f8-8a5e-c9d7a39fb637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:48:27.203528 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbebcc7-e1b3-43f8-8a5e-c9d7a39fb637 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:27.203811 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbebcc7-e1b3-43f8-8a5e-c9d7a39fb637 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:27.204115 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbebcc7-e1b3-43f8-8a5e-c9d7a39fb637 None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:27.204889 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbebcc7-e1b3-43f8-8a5e-c9d7a39fb637 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:48:27.205193 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbebcc7-e1b3-43f8-8a5e-c9d7a39fb637 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:48:27.206607 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbebcc7-e1b3-43f8-8a5e-c9d7a39fb637 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:48:27.218895 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 633/1272] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Jun 10 12:48:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:48:27.224878 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575a2647-2869-4ba2-b541-f1fe1365b1da None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:48:27.314161 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:48:27.322926 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575a2647-2869-4ba2-b541-f1fe1365b1da None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:48:27.356567 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074dad31-fae2-4039-a91a-dd6dcaf6a495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:48:27.357447 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074dad31-fae2-4039-a91a-dd6dcaf6a495 None admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:27.357738 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074dad31-fae2-4039-a91a-dd6dcaf6a495 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:27.358008 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074dad31-fae2-4039-a91a-dd6dcaf6a495 None admin] PATH_INFO: `/v3/projects/73ea5b23a2d34544ab493f4c5e4034b8/users/d972ab782b3f4d2aba575314f16dafc7/roles/b43d5a8247e244869c2c70f986f42e8a` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:27.359990 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074dad31-fae2-4039-a91a-dd6dcaf6a495 None admin] RBAC: Authorizing `identity:create_grant(project_id=73ea5b23a2d34544ab493f4c5e4034b8, user_id=d972ab782b3f4d2aba575314f16dafc7, role_id=b43d5a8247e244869c2c70f986f42e8a)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:48:27.361457 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074dad31-fae2-4039-a91a-dd6dcaf6a495 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:48:27.466004 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 640/1273] 10.222.0.27 () {58 vars in 1439 bytes} [Fri Jun 10 12:48:27 2022] PUT /identity/v3/projects/73ea5b23a2d34544ab493f4c5e4034b8/users/d972ab782b3f4d2aba575314f16dafc7/roles/b43d5a8247e244869c2c70f986f42e8a => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 12:48:29.549068 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade27c28-2eaf-49ca-8cac-52668b2e9dab None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:29.549602 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade27c28-2eaf-49ca-8cac-52668b2e9dab None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:29.549602 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade27c28-2eaf-49ca-8cac-52668b2e9dab None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:29.550979 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 634/1274] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:48:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:48:29.556057 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc66c579-5302-4072-83d0-a5141effc431 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:29.556447 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc66c579-5302-4072-83d0-a5141effc431 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:29.556625 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc66c579-5302-4072-83d0-a5141effc431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:29.572653 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-fc66c579-5302-4072-83d0-a5141effc431 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:48:29.573837 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-fc66c579-5302-4072-83d0-a5141effc431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:48:29.638573 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 641/1275] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:48:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 10 12:48:29.663530 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade27c28-2eaf-49ca-8cac-52668b2e9dab None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:48:29.763259 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade27c28-2eaf-49ca-8cac-52668b2e9dab None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:48:29.781090 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:48:29.796821 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b47d18-82cc-4de1-b1f3-c34218213253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:48:29.797605 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b47d18-82cc-4de1-b1f3-c34218213253 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:29.797887 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b47d18-82cc-4de1-b1f3-c34218213253 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:29.798158 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b47d18-82cc-4de1-b1f3-c34218213253 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:29.807905 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b47d18-82cc-4de1-b1f3-c34218213253 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 10 12:48:29.815341 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-d6b47d18-82cc-4de1-b1f3-c34218213253 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 10 12:48:29.815341 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:48:29.815341 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:48:29.815341 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:48:29.815341 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:48:29.815341 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:48:29.815341 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:48:29.815341 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:48:29.815341 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:48:29.815341 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:48:29.815341 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:48:29.815341 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:48:29.815341 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 10 12:48:29.815341 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 10 12:48:29.815341 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 10 12:48:29.815341 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 10 12:48:29.816510 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 10 12:48:29.816510 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 10 12:48:29.816510 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 10 12:48:29.816510 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 10 12:48:29.816510 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:48:29.816510 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:48:29.816510 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:48:29.816510 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:48:29.816510 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:48:29.816510 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:48:29.816510 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:48:29.816510 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:48:29.816510 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:48:29.816510 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:48:29.816510 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:48:29.816510 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:48:29.817739 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:48:29.817739 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:48:29.817739 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:48:29.817739 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:48:29.817739 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 10 12:48:29.817739 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 10 12:48:29.817739 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 10 12:48:29.817739 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 10 12:48:29.817739 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 10 12:48:29.817739 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 10 12:48:29.817739 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 10 12:48:29.817739 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:48:29.817739 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 635/1276] 10.222.0.27 () {56 vars in 1096 bytes} [Fri Jun 10 12:48:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:48:29.822486 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc66c579-5302-4072-83d0-a5141effc431 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:48:29.902669 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:48:29.919816 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc66c579-5302-4072-83d0-a5141effc431 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:48:29.952796 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d3f620-e70d-4a4d-9687-12204a76ecc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:48:29.953457 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d3f620-e70d-4a4d-9687-12204a76ecc3 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:29.953741 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d3f620-e70d-4a4d-9687-12204a76ecc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:29.954020 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d3f620-e70d-4a4d-9687-12204a76ecc3 None admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:29.954759 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d3f620-e70d-4a4d-9687-12204a76ecc3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:48:29.955068 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d3f620-e70d-4a4d-9687-12204a76ecc3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:48:29.956522 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d3f620-e70d-4a4d-9687-12204a76ecc3 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:48:29.964980 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 642/1277] 10.222.0.27 () {56 vars in 1099 bytes} [Fri Jun 10 12:48:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:48:29.970907 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b47d18-82cc-4de1-b1f3-c34218213253 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:48:30.070270 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b47d18-82cc-4de1-b1f3-c34218213253 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:48:30.102087 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742c79bf-7c9a-40bf-bfa5-d47f53b9d0e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:48:30.102857 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742c79bf-7c9a-40bf-bfa5-d47f53b9d0e7 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:30.103135 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742c79bf-7c9a-40bf-bfa5-d47f53b9d0e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:30.103403 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742c79bf-7c9a-40bf-bfa5-d47f53b9d0e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:30.113446 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742c79bf-7c9a-40bf-bfa5-d47f53b9d0e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:48:30.114875 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742c79bf-7c9a-40bf-bfa5-d47f53b9d0e7 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:48:30.131743 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-742c79bf-7c9a-40bf-bfa5-d47f53b9d0e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:48:30.131743 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:48:30.131743 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:48:30.131743 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:48:30.131743 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:48:30.131743 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:48:30.131743 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:48:30.131743 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:48:30.131743 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:48:30.131743 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:48:30.131743 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:48:30.131743 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:48:30.131743 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 10 12:48:30.131743 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 10 12:48:30.131743 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 10 12:48:30.132839 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 10 12:48:30.132839 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:48:30.132839 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:48:30.132839 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:48:30.132839 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:48:30.132839 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:48:30.132839 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:48:30.132839 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:48:30.132839 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:48:30.132839 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:48:30.132839 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:48:30.132839 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:48:30.132839 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:48:30.132839 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:48:30.132839 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:48:30.132839 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:48:30.132839 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:48:30.133813 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 10 12:48:30.133813 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 10 12:48:30.133813 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:48:30.133813 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:48:30.133813 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 636/1278] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:48:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:48:30.138614 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d3f620-e70d-4a4d-9687-12204a76ecc3 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:48:30.235829 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d3f620-e70d-4a4d-9687-12204a76ecc3 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:48:30.267922 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352db4a0-c7c9-4a62-b641-8389b09973de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:48:30.268578 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352db4a0-c7c9-4a62-b641-8389b09973de None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:30.268864 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352db4a0-c7c9-4a62-b641-8389b09973de None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:30.269144 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352db4a0-c7c9-4a62-b641-8389b09973de None admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:30.269865 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352db4a0-c7c9-4a62-b641-8389b09973de None admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:48:30.270171 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352db4a0-c7c9-4a62-b641-8389b09973de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:48:30.271523 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352db4a0-c7c9-4a62-b641-8389b09973de None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:48:30.283176 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 643/1279] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Jun 10 12:48:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:48:30.288728 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742c79bf-7c9a-40bf-bfa5-d47f53b9d0e7 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:48:30.320193 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:48:30.386802 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742c79bf-7c9a-40bf-bfa5-d47f53b9d0e7 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:48:30.418735 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a65a2c-299b-4b31-b9d6-869dc6696553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:48:30.419511 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a65a2c-299b-4b31-b9d6-869dc6696553 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:30.419791 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a65a2c-299b-4b31-b9d6-869dc6696553 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:30.420092 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a65a2c-299b-4b31-b9d6-869dc6696553 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:30.427299 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a65a2c-299b-4b31-b9d6-869dc6696553 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:48:30.428759 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a65a2c-299b-4b31-b9d6-869dc6696553 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:48:30.443429 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-69a65a2c-299b-4b31-b9d6-869dc6696553 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 10 12:48:30.443429 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:48:30.443429 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:48:30.443429 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:48:30.443429 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:48:30.443429 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:48:30.443429 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:48:30.443429 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:48:30.443429 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:48:30.443429 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:48:30.443429 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:48:30.443429 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:48:30.443429 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 10 12:48:30.443429 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 10 12:48:30.443429 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 10 12:48:30.443429 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 10 12:48:30.444497 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:48:30.444497 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:48:30.444497 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 10 12:48:30.444497 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 10 12:48:30.444497 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 10 12:48:30.444497 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 10 12:48:30.444497 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:48:30.444497 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:48:30.444497 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:48:30.444497 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:48:30.444497 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:48:30.444497 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:48:30.444497 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:48:30.444497 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:48:30.444497 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:48:30.444497 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:48:30.444497 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:48:30.444497 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:48:30.445765 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:48:30.445765 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:48:30.445765 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 10 12:48:30.445765 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 10 12:48:30.445765 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 10 12:48:30.445765 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 10 12:48:30.445765 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 10 12:48:30.445765 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 10 12:48:30.445765 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 10 12:48:30.445765 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:48:30.445765 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 637/1280] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:48:30 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:48:30.450521 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352db4a0-c7c9-4a62-b641-8389b09973de None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:48:30.454400 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:48:30.548977 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352db4a0-c7c9-4a62-b641-8389b09973de None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:48:30.581738 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2d10a4-eea3-4b8c-a4b8-1854c53915ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:48:30.582399 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2d10a4-eea3-4b8c-a4b8-1854c53915ee None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:30.582681 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2d10a4-eea3-4b8c-a4b8-1854c53915ee None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:30.582957 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2d10a4-eea3-4b8c-a4b8-1854c53915ee None admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:30.583783 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2d10a4-eea3-4b8c-a4b8-1854c53915ee None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:48:30.584114 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2d10a4-eea3-4b8c-a4b8-1854c53915ee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:48:30.585526 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2d10a4-eea3-4b8c-a4b8-1854c53915ee None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:48:30.602367 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 644/1281] 10.222.0.27 () {56 vars in 1143 bytes} [Fri Jun 10 12:48:30 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:48:30.608995 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a65a2c-299b-4b31-b9d6-869dc6696553 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:48:30.697313 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:48:30.707702 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a65a2c-299b-4b31-b9d6-869dc6696553 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:48:30.746800 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40161903-7723-42e7-8970-c7ee6a91925c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:48:30.747451 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40161903-7723-42e7-8970-c7ee6a91925c None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:30.747740 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40161903-7723-42e7-8970-c7ee6a91925c None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:30.748053 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40161903-7723-42e7-8970-c7ee6a91925c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:30.814203 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40161903-7723-42e7-8970-c7ee6a91925c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:48:30.815640 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40161903-7723-42e7-8970-c7ee6a91925c None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:48:30.903446 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:48:30.913985 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 638/1282] 10.222.0.27 () {58 vars in 1282 bytes} [Fri Jun 10 12:48:30 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 10 12:48:30.920003 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2d10a4-eea3-4b8c-a4b8-1854c53915ee None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:48:30.982347 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:48:31.020706 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2d10a4-eea3-4b8c-a4b8-1854c53915ee None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:48:31.053212 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a453ae1-9371-4512-967f-0dbff1016466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:48:31.053993 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a453ae1-9371-4512-967f-0dbff1016466 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:31.054276 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a453ae1-9371-4512-967f-0dbff1016466 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:31.054550 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a453ae1-9371-4512-967f-0dbff1016466 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:31.064180 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a453ae1-9371-4512-967f-0dbff1016466 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:48:31.065599 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a453ae1-9371-4512-967f-0dbff1016466 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:48:31.083105 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-3a453ae1-9371-4512-967f-0dbff1016466 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:48:31.083105 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:48:31.083105 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:48:31.083105 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:48:31.083105 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:48:31.083105 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:48:31.083105 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:48:31.083105 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:48:31.083105 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:48:31.083105 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:48:31.083105 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:48:31.083105 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:48:31.083105 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 10 12:48:31.083105 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 10 12:48:31.083105 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 10 12:48:31.084240 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 10 12:48:31.084240 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:48:31.084240 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:48:31.084240 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:48:31.084240 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:48:31.084240 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:48:31.084240 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:48:31.084240 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:48:31.084240 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:48:31.084240 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:48:31.084240 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:48:31.084240 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:48:31.084240 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:48:31.084240 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:48:31.084240 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:48:31.084240 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:48:31.084240 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:48:31.085325 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 10 12:48:31.085325 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 10 12:48:31.085325 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 10 12:48:31.085325 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:48:31.085325 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 645/1283] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:48:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:48:31.090333 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40161903-7723-42e7-8970-c7ee6a91925c None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:48:31.193232 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40161903-7723-42e7-8970-c7ee6a91925c None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:48:31.229815 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3431f0-62b8-408b-bddc-775968dc57d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:48:31.230495 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3431f0-62b8-408b-bddc-775968dc57d7 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:31.230780 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3431f0-62b8-408b-bddc-775968dc57d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:31.231061 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3431f0-62b8-408b-bddc-775968dc57d7 None admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:31.231800 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3431f0-62b8-408b-bddc-775968dc57d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:48:31.235641 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3431f0-62b8-408b-bddc-775968dc57d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:48:31.235641 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3431f0-62b8-408b-bddc-775968dc57d7 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:48:31.246813 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 639/1284] 10.222.0.27 () {56 vars in 1109 bytes} [Fri Jun 10 12:48:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:48:31.252732 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a453ae1-9371-4512-967f-0dbff1016466 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:48:31.358290 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a453ae1-9371-4512-967f-0dbff1016466 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:48:31.373242 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:48:31.394860 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8b675d-3819-46ed-8ceb-07392e8c8da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:48:31.395669 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8b675d-3819-46ed-8ceb-07392e8c8da5 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:31.395972 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8b675d-3819-46ed-8ceb-07392e8c8da5 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:31.396249 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8b675d-3819-46ed-8ceb-07392e8c8da5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:31.403369 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8b675d-3819-46ed-8ceb-07392e8c8da5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:48:31.404887 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8b675d-3819-46ed-8ceb-07392e8c8da5 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:48:31.420475 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application [None req-6c8b675d-3819-46ed-8ceb-07392e8c8da5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 10 12:48:31.420475 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:48:31.420475 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:48:31.420475 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:48:31.420475 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:48:31.420475 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:48:31.420475 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:48:31.420475 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:48:31.420475 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:48:31.420475 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:48:31.420475 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:48:31.420475 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:48:31.420475 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 10 12:48:31.420475 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 10 12:48:31.420475 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 10 12:48:31.421784 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 10 12:48:31.421784 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:48:31.421784 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:48:31.421784 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:48:31.421784 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:48:31.421784 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:48:31.421784 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:48:31.421784 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:48:31.421784 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application with Lock( Jun 10 12:48:31.421784 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:48:31.421784 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:48:31.421784 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:48:31.421784 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:48:31.421784 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:48:31.421784 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:48:31.421784 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:48:31.421784 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:48:31.423252 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 10 12:48:31.423252 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 10 12:48:31.423252 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 10 12:48:31.423252 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 10 12:48:31.423252 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 10 12:48:31.423252 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 10 12:48:31.423252 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 10 12:48:31.423252 dv-cinder-master-3-smb devstack@keystone.service[80237]: ERROR keystone.server.flask.application Jun 10 12:48:31.423252 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 646/1285] 10.222.0.27 () {56 vars in 1111 bytes} [Fri Jun 10 12:48:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:48:31.428849 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3431f0-62b8-408b-bddc-775968dc57d7 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:48:31.484462 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:48:31.534735 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3431f0-62b8-408b-bddc-775968dc57d7 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:48:31.569973 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1bae42-e296-4e3d-8dd9-b0ef11ea1391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:48:31.570644 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1bae42-e296-4e3d-8dd9-b0ef11ea1391 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:31.570937 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1bae42-e296-4e3d-8dd9-b0ef11ea1391 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:31.571218 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1bae42-e296-4e3d-8dd9-b0ef11ea1391 None admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:31.572052 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1bae42-e296-4e3d-8dd9-b0ef11ea1391 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:48:31.572370 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1bae42-e296-4e3d-8dd9-b0ef11ea1391 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:48:31.573811 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1bae42-e296-4e3d-8dd9-b0ef11ea1391 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:48:31.586910 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 640/1286] 10.222.0.27 () {56 vars in 1148 bytes} [Fri Jun 10 12:48:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:48:31.593169 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8b675d-3819-46ed-8ceb-07392e8c8da5 None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:48:31.692181 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8b675d-3819-46ed-8ceb-07392e8c8da5 None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:48:31.725745 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dbf734-3e63-4d60-b051-27220fb95004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:48:31.726394 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dbf734-3e63-4d60-b051-27220fb95004 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:31.726683 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dbf734-3e63-4d60-b051-27220fb95004 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:31.726967 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dbf734-3e63-4d60-b051-27220fb95004 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:31.727792 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dbf734-3e63-4d60-b051-27220fb95004 None admin] RBAC: Adding query filter params (role.id=b43d5a8247e244869c2c70f986f42e8a, scope.project.id=73ea5b23a2d34544ab493f4c5e4034b8, user.id=d972ab782b3f4d2aba575314f16dafc7) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:48:31.728137 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dbf734-3e63-4d60-b051-27220fb95004 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:48:31.729602 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dbf734-3e63-4d60-b051-27220fb95004 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:48:31.735901 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 647/1287] 10.222.0.27 () {56 vars in 1374 bytes} [Fri Jun 10 12:48:31 2022] GET /identity/v3/role_assignments?user.id=d972ab782b3f4d2aba575314f16dafc7&scope.project.id=73ea5b23a2d34544ab493f4c5e4034b8&role.id=b43d5a8247e244869c2c70f986f42e8a => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:48:33.861733 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6f95be-31c8-494a-b9ee-1b77babc6ce5 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:33.862253 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6f95be-31c8-494a-b9ee-1b77babc6ce5 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:33.862376 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6f95be-31c8-494a-b9ee-1b77babc6ce5 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:33.863695 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 641/1288] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:48:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:48:33.868911 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a540e3b-4393-4ab3-8fb6-9cb75c25dd9b None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:33.869289 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a540e3b-4393-4ab3-8fb6-9cb75c25dd9b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:33.869470 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a540e3b-4393-4ab3-8fb6-9cb75c25dd9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:33.886302 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-6a540e3b-4393-4ab3-8fb6-9cb75c25dd9b None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:48:33.887466 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6a540e3b-4393-4ab3-8fb6-9cb75c25dd9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:48:33.916392 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:48:33.953478 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 648/1289] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:48:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 10 12:48:33.974617 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6f95be-31c8-494a-b9ee-1b77babc6ce5 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:48:34.074041 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6f95be-31c8-494a-b9ee-1b77babc6ce5 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:48:34.084368 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:48:34.106257 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0aaa65-671e-4a95-a901-2fd40d1a6235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:48:34.107044 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0aaa65-671e-4a95-a901-2fd40d1a6235 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:34.112016 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0aaa65-671e-4a95-a901-2fd40d1a6235 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:34.112016 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0aaa65-671e-4a95-a901-2fd40d1a6235 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:34.112016 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0aaa65-671e-4a95-a901-2fd40d1a6235 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:48:34.112016 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0aaa65-671e-4a95-a901-2fd40d1a6235 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:48:34.121565 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-cb0aaa65-671e-4a95-a901-2fd40d1a6235 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 10 12:48:34.121565 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:48:34.121565 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 10 12:48:34.121565 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 10 12:48:34.121565 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 10 12:48:34.121565 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 10 12:48:34.121565 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 10 12:48:34.121565 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 10 12:48:34.121565 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 10 12:48:34.121565 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:48:34.121565 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 10 12:48:34.121565 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:48:34.121565 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:48:34.121565 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:48:34.121565 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:48:34.121565 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:48:34.121565 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:48:34.127760 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:48:34.127760 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:48:34.127760 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:48:34.127760 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:48:34.127760 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:48:34.127760 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:48:34.127760 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 10 12:48:34.127760 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 10 12:48:34.127760 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 10 12:48:34.127760 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 10 12:48:34.127760 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:48:34.127760 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:48:34.127760 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:48:34.127760 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:48:34.127760 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:48:34.127760 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:48:34.128781 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:48:34.128781 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:48:34.128781 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:48:34.128781 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:48:34.128781 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:48:34.128781 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:48:34.128781 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:48:34.128781 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:48:34.128781 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:48:34.128781 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:48:34.128781 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 10 12:48:34.128781 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 10 12:48:34.128781 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 10 12:48:34.128781 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:48:34.128781 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 642/1290] 10.222.0.27 () {56 vars in 1117 bytes} [Fri Jun 10 12:48:33 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:48:34.129749 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a540e3b-4393-4ab3-8fb6-9cb75c25dd9b None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:48:34.224154 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a540e3b-4393-4ab3-8fb6-9cb75c25dd9b None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:48:34.257665 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dc1916-5987-4d66-929c-30dd0290c219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:48:34.258323 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dc1916-5987-4d66-929c-30dd0290c219 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:34.258614 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dc1916-5987-4d66-929c-30dd0290c219 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:34.258892 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dc1916-5987-4d66-929c-30dd0290c219 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:34.259653 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dc1916-5987-4d66-929c-30dd0290c219 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:48:34.259991 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dc1916-5987-4d66-929c-30dd0290c219 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:48:34.261347 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dc1916-5987-4d66-929c-30dd0290c219 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:48:34.266138 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 649/1291] 10.222.0.27 () {56 vars in 1116 bytes} [Fri Jun 10 12:48:34 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:48:34.271818 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0aaa65-671e-4a95-a901-2fd40d1a6235 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:48:34.370403 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0aaa65-671e-4a95-a901-2fd40d1a6235 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:48:34.403729 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa4b13d-6d69-43e9-92c0-6b663fbc5c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:48:34.404418 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa4b13d-6d69-43e9-92c0-6b663fbc5c7f None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:34.405508 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa4b13d-6d69-43e9-92c0-6b663fbc5c7f None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:34.405782 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa4b13d-6d69-43e9-92c0-6b663fbc5c7f None admin] PATH_INFO: `/v3/services` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:34.406513 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa4b13d-6d69-43e9-92c0-6b663fbc5c7f None admin] RBAC: Adding query filter params (type=placement) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:48:34.406815 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa4b13d-6d69-43e9-92c0-6b663fbc5c7f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:48:34.408196 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa4b13d-6d69-43e9-92c0-6b663fbc5c7f None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:48:34.412695 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 643/1292] 10.222.0.27 () {56 vars in 1116 bytes} [Fri Jun 10 12:48:34 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:48:36.493011 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a03504-7cf6-4651-ae8d-6efbec0e3bd6 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:36.493581 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a03504-7cf6-4651-ae8d-6efbec0e3bd6 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:36.493581 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a03504-7cf6-4651-ae8d-6efbec0e3bd6 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:36.494965 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 650/1293] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:48:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:48:36.500015 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2d2988-8dfe-458c-a30b-bf6f325c1fe4 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:36.500433 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2d2988-8dfe-458c-a30b-bf6f325c1fe4 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:36.500572 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2d2988-8dfe-458c-a30b-bf6f325c1fe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:36.517356 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-2b2d2988-8dfe-458c-a30b-bf6f325c1fe4 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:48:36.518509 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-2b2d2988-8dfe-458c-a30b-bf6f325c1fe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:48:36.547194 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:48:36.582003 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 644/1294] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:48:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 10 12:48:36.604523 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a03504-7cf6-4651-ae8d-6efbec0e3bd6 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:48:36.635357 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:48:36.702732 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a03504-7cf6-4651-ae8d-6efbec0e3bd6 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:48:36.734402 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e146ade2-36ae-4da8-8966-e5f8f3488ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:48:36.735044 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e146ade2-36ae-4da8-8966-e5f8f3488ee0 None admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:36.735323 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e146ade2-36ae-4da8-8966-e5f8f3488ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:36.735598 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e146ade2-36ae-4da8-8966-e5f8f3488ee0 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:36.736345 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e146ade2-36ae-4da8-8966-e5f8f3488ee0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:48:36.737722 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e146ade2-36ae-4da8-8966-e5f8f3488ee0 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:48:36.819908 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 651/1295] 10.222.0.27 () {60 vars in 1141 bytes} [Fri Jun 10 12:48:36 2022] POST /identity/v3/services => generated 246 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 10 12:48:38.939780 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ddee73-f69c-4440-85bb-ffd6c7a38ceb None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:38.940330 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ddee73-f69c-4440-85bb-ffd6c7a38ceb None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:38.940330 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ddee73-f69c-4440-85bb-ffd6c7a38ceb None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:38.941704 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 645/1296] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:48:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:48:38.946743 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be732780-7be8-4183-833c-575e0d8b4e4c None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:38.947141 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be732780-7be8-4183-833c-575e0d8b4e4c None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:38.947301 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be732780-7be8-4183-833c-575e0d8b4e4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:38.963774 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-be732780-7be8-4183-833c-575e0d8b4e4c None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:48:38.964978 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-be732780-7be8-4183-833c-575e0d8b4e4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:48:39.035850 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 652/1297] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:48:38 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 10 12:48:39.058428 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ddee73-f69c-4440-85bb-ffd6c7a38ceb None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:48:39.156547 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ddee73-f69c-4440-85bb-ffd6c7a38ceb None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:48:39.189704 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787903f2-96ff-48cf-93d1-2d2c407d1387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:48:39.190513 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787903f2-96ff-48cf-93d1-2d2c407d1387 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:39.190792 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787903f2-96ff-48cf-93d1-2d2c407d1387 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:39.191064 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787903f2-96ff-48cf-93d1-2d2c407d1387 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:39.191704 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787903f2-96ff-48cf-93d1-2d2c407d1387 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:48:39.193077 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787903f2-96ff-48cf-93d1-2d2c407d1387 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:48:39.204540 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-787903f2-96ff-48cf-93d1-2d2c407d1387 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 10 12:48:39.204540 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:48:39.204540 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 10 12:48:39.204540 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 10 12:48:39.204540 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 10 12:48:39.204540 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 10 12:48:39.204540 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 10 12:48:39.204540 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 10 12:48:39.204540 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 10 12:48:39.204540 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:48:39.204540 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 10 12:48:39.204540 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:48:39.204540 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:48:39.204540 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:48:39.204540 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:48:39.204540 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:48:39.204540 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:48:39.205748 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:48:39.205748 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:48:39.205748 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:48:39.205748 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:48:39.205748 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:48:39.205748 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:48:39.205748 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 10 12:48:39.205748 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 10 12:48:39.205748 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 10 12:48:39.205748 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 10 12:48:39.205748 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:48:39.205748 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:48:39.205748 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:48:39.205748 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:48:39.205748 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:48:39.205748 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:48:39.207025 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:48:39.207025 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:48:39.207025 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:48:39.207025 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:48:39.207025 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:48:39.207025 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:48:39.207025 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:48:39.207025 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:48:39.207025 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:48:39.207025 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:48:39.207025 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 10 12:48:39.207025 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 10 12:48:39.207025 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 10 12:48:39.207025 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:48:39.207025 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 646/1298] 10.222.0.27 () {56 vars in 1117 bytes} [Fri Jun 10 12:48:39 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:48:39.211757 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be732780-7be8-4183-833c-575e0d8b4e4c None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:48:39.230297 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:48:39.308359 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be732780-7be8-4183-833c-575e0d8b4e4c None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:48:39.342600 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048f13c3-af90-4abd-ae6e-61eabd96043f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:48:39.343252 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048f13c3-af90-4abd-ae6e-61eabd96043f None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:39.343538 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048f13c3-af90-4abd-ae6e-61eabd96043f None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:39.343817 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048f13c3-af90-4abd-ae6e-61eabd96043f None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:39.344601 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048f13c3-af90-4abd-ae6e-61eabd96043f None admin] RBAC: Adding query filter params (name=placement) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:48:39.344906 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048f13c3-af90-4abd-ae6e-61eabd96043f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:48:39.346258 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048f13c3-af90-4abd-ae6e-61eabd96043f None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:48:39.351323 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 653/1299] 10.222.0.27 () {56 vars in 1116 bytes} [Fri Jun 10 12:48:39 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:48:39.357308 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787903f2-96ff-48cf-93d1-2d2c407d1387 None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:48:39.387894 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:48:39.460044 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787903f2-96ff-48cf-93d1-2d2c407d1387 None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:48:39.493899 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b791ce06-ed9d-443e-afac-5cefc61c61f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:48:39.494546 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b791ce06-ed9d-443e-afac-5cefc61c61f2 None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:39.494832 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b791ce06-ed9d-443e-afac-5cefc61c61f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:39.495169 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b791ce06-ed9d-443e-afac-5cefc61c61f2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:39.495997 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b791ce06-ed9d-443e-afac-5cefc61c61f2 None admin] RBAC: Adding query filter params (interface=public, service_id=338411c5ea2642f09bceb4ad7d9c57d1, region_id=RegionOne) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:48:39.496318 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b791ce06-ed9d-443e-afac-5cefc61c61f2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:48:39.497708 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b791ce06-ed9d-443e-afac-5cefc61c61f2 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:48:39.502794 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 647/1300] 10.222.0.27 () {56 vars in 1251 bytes} [Fri Jun 10 12:48:39 2022] GET /identity/v3/endpoints?service_id=338411c5ea2642f09bceb4ad7d9c57d1&interface=public®ion_id=RegionOne => generated 194 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:48:39.508419 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048f13c3-af90-4abd-ae6e-61eabd96043f None admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:48:39.587994 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:48:39.605584 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048f13c3-af90-4abd-ae6e-61eabd96043f None admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:48:39.638118 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed35e41-8a62-47fd-9de2-4bf8fea46823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:48:39.639353 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed35e41-8a62-47fd-9de2-4bf8fea46823 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:39.639647 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed35e41-8a62-47fd-9de2-4bf8fea46823 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:39.639952 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed35e41-8a62-47fd-9de2-4bf8fea46823 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:39.640691 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed35e41-8a62-47fd-9de2-4bf8fea46823 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:48:39.642042 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed35e41-8a62-47fd-9de2-4bf8fea46823 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:48:39.648657 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 654/1301] 10.222.0.27 () {56 vars in 1087 bytes} [Fri Jun 10 12:48:39 2022] GET /identity/v3/services => generated 2191 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 12:48:41.752304 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644d803f-d076-499c-b7dc-0a2fa96fabd1 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:41.752881 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644d803f-d076-499c-b7dc-0a2fa96fabd1 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:41.752881 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644d803f-d076-499c-b7dc-0a2fa96fabd1 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:41.754213 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 648/1302] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:48:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:48:41.759378 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1609e768-db81-4643-8565-69600538d63f None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:41.759786 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1609e768-db81-4643-8565-69600538d63f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:41.759986 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1609e768-db81-4643-8565-69600538d63f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:41.776093 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-1609e768-db81-4643-8565-69600538d63f None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:48:41.777303 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1609e768-db81-4643-8565-69600538d63f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:48:41.842942 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 655/1303] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:48:41 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 10 12:48:41.865994 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644d803f-d076-499c-b7dc-0a2fa96fabd1 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:48:41.959336 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:48:41.965879 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644d803f-d076-499c-b7dc-0a2fa96fabd1 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:48:41.999103 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fc215f-696a-41db-8507-303f2e236d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:48:41.999961 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fc215f-696a-41db-8507-303f2e236d8b None admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:42.000244 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fc215f-696a-41db-8507-303f2e236d8b None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:42.000516 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fc215f-696a-41db-8507-303f2e236d8b None admin] PATH_INFO: `/v3/services/placement` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:42.001158 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fc215f-696a-41db-8507-303f2e236d8b None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:48:42.002613 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fc215f-696a-41db-8507-303f2e236d8b None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:48:42.013874 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application [None req-61fc215f-696a-41db-8507-303f2e236d8b None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 10 12:48:42.013874 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:48:42.013874 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 10 12:48:42.013874 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 10 12:48:42.013874 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 10 12:48:42.013874 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 10 12:48:42.013874 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 10 12:48:42.013874 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 10 12:48:42.013874 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 10 12:48:42.013874 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:48:42.013874 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 10 12:48:42.013874 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:48:42.013874 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 10 12:48:42.013874 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 10 12:48:42.013874 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 10 12:48:42.013874 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 10 12:48:42.013874 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 10 12:48:42.015067 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 10 12:48:42.015067 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 10 12:48:42.015067 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 10 12:48:42.015067 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 10 12:48:42.015067 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 10 12:48:42.015067 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 10 12:48:42.015067 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 10 12:48:42.015067 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 10 12:48:42.015067 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 10 12:48:42.015067 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 10 12:48:42.015067 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 10 12:48:42.015067 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 10 12:48:42.015067 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 10 12:48:42.015067 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 10 12:48:42.015067 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 10 12:48:42.015067 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.get_or_create( Jun 10 12:48:42.016206 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 10 12:48:42.016206 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application with Lock( Jun 10 12:48:42.016206 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 10 12:48:42.016206 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self._enter() Jun 10 12:48:42.016206 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 10 12:48:42.016206 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 10 12:48:42.016206 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 10 12:48:42.016206 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application return self.creator() Jun 10 12:48:42.016206 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 10 12:48:42.016206 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application created_value = creator( Jun 10 12:48:42.016206 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 10 12:48:42.016206 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 10 12:48:42.016206 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 10 12:48:42.016206 dv-cinder-master-3-smb devstack@keystone.service[80238]: ERROR keystone.server.flask.application Jun 10 12:48:42.016206 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 649/1304] 10.222.0.27 () {56 vars in 1117 bytes} [Fri Jun 10 12:48:41 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 10 12:48:42.021164 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1609e768-db81-4643-8565-69600538d63f None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:48:42.098505 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:48:42.121290 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1609e768-db81-4643-8565-69600538d63f None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:48:42.152920 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b932299-807b-4f7e-ac12-0bf65542f908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:48:42.153603 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b932299-807b-4f7e-ac12-0bf65542f908 None admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:42.153892 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b932299-807b-4f7e-ac12-0bf65542f908 None admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:42.154680 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b932299-807b-4f7e-ac12-0bf65542f908 None admin] PATH_INFO: `/v3/services` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:42.155675 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b932299-807b-4f7e-ac12-0bf65542f908 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:48:42.156015 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b932299-807b-4f7e-ac12-0bf65542f908 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:48:42.157371 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b932299-807b-4f7e-ac12-0bf65542f908 None admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:48:42.162331 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 656/1305] 10.222.0.27 () {56 vars in 1116 bytes} [Fri Jun 10 12:48:42 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 12:48:42.168330 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fc215f-696a-41db-8507-303f2e236d8b None admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:48:42.265706 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fc215f-696a-41db-8507-303f2e236d8b None admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:48:42.298600 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0ff150-a542-48bb-b365-ad13fba56239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:48:42.299249 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0ff150-a542-48bb-b365-ad13fba56239 None admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:48:42.299538 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0ff150-a542-48bb-b365-ad13fba56239 None admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:48:42.299894 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0ff150-a542-48bb-b365-ad13fba56239 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:48:42.300642 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0ff150-a542-48bb-b365-ad13fba56239 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:48:42.302012 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0ff150-a542-48bb-b365-ad13fba56239 None admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:48:42.380385 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 650/1306] 10.222.0.27 () {60 vars in 1144 bytes} [Fri Jun 10 12:48:42 2022] POST /identity/v3/endpoints => generated 331 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 12:48:52.168746 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=30 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:48:52.375593 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:54:32.474951 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda854bb-449c-4129-9528-614381ad852c None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:54:32.476028 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda854bb-449c-4129-9528-614381ad852c None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:54:32.476536 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda854bb-449c-4129-9528-614381ad852c None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:54:32.481943 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 657/1307] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:54:32 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:54:32.483303 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8480bd63-e7c8-4504-b98d-17e3d8bc0a92 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:54:32.483721 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8480bd63-e7c8-4504-b98d-17e3d8bc0a92 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:54:32.483971 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8480bd63-e7c8-4504-b98d-17e3d8bc0a92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:54:32.557424 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-8480bd63-e7c8-4504-b98d-17e3d8bc0a92 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:54:32.558640 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-8480bd63-e7c8-4504-b98d-17e3d8bc0a92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:54:32.680895 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 651/1308] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:54:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 10 12:54:32.727880 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cc9dbf-5756-42b6-a345-8e72a7202199 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:54:32.728236 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cc9dbf-5756-42b6-a345-8e72a7202199 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:54:32.728523 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cc9dbf-5756-42b6-a345-8e72a7202199 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:54:32.730383 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 658/1309] 10.222.0.27 () {54 vars in 964 bytes} [Fri Jun 10 12:54:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:54:32.740935 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27b1ab9-827e-47c7-9bb2-518c84835c87 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:54:32.741255 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27b1ab9-827e-47c7-9bb2-518c84835c87 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:54:32.741524 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27b1ab9-827e-47c7-9bb2-518c84835c87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:54:32.830199 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-e27b1ab9-827e-47c7-9bb2-518c84835c87 None None] MFA Rules not processed for user `bc4415461a974c24ad7105a97c7d915b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:54:32.831325 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e27b1ab9-827e-47c7-9bb2-518c84835c87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:54:32.857961 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:54:32.913249 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 652/1310] 10.222.0.27 () {58 vars in 1061 bytes} [Fri Jun 10 12:54:32 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 12:54:32.922465 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cc9dbf-5756-42b6-a345-8e72a7202199 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:54:32.924317 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c7cc9dbf-5756-42b6-a345-8e72a7202199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:54:33.047614 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cc9dbf-5756-42b6-a345-8e72a7202199 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:54:33.124165 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a85ab0-9894-4398-a2fe-4022d83e1022 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:54:33.125194 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a85ab0-9894-4398-a2fe-4022d83e1022 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:54:33.125194 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a85ab0-9894-4398-a2fe-4022d83e1022 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:54:33.125395 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a85ab0-9894-4398-a2fe-4022d83e1022 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:54:33.127635 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-40a85ab0-9894-4398-a2fe-4022d83e1022 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:54:33.183652 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:54:33.239162 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a85ab0-9894-4398-a2fe-4022d83e1022 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:54:33.240539 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a85ab0-9894-4398-a2fe-4022d83e1022 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:54:33.260088 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 659/1311] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 12:54:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 12:54:35.666340 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2afdd8-1cdd-445d-bbd7-aefb69ec7e01 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:54:35.666877 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2afdd8-1cdd-445d-bbd7-aefb69ec7e01 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:54:35.666877 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2afdd8-1cdd-445d-bbd7-aefb69ec7e01 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:54:35.668301 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 653/1312] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:54:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:54:35.673956 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f94580-98ae-4e94-97fe-3b237c19a718 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:54:35.674397 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f94580-98ae-4e94-97fe-3b237c19a718 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:54:35.674554 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f94580-98ae-4e94-97fe-3b237c19a718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:54:35.756798 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-01f94580-98ae-4e94-97fe-3b237c19a718 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:54:35.757995 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-01f94580-98ae-4e94-97fe-3b237c19a718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:54:35.855117 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 660/1313] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:54:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 12:54:35.892842 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2afdd8-1cdd-445d-bbd7-aefb69ec7e01 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:54:35.894405 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-6a2afdd8-1cdd-445d-bbd7-aefb69ec7e01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:54:36.005741 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2afdd8-1cdd-445d-bbd7-aefb69ec7e01 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:54:36.078922 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:54:36.081812 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0804964a-c388-4112-93b9-3c107b02a7c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:54:36.082459 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0804964a-c388-4112-93b9-3c107b02a7c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:54:36.082743 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0804964a-c388-4112-93b9-3c107b02a7c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:54:36.083011 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0804964a-c388-4112-93b9-3c107b02a7c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:54:36.084682 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0804964a-c388-4112-93b9-3c107b02a7c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:54:36.196599 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0804964a-c388-4112-93b9-3c107b02a7c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:54:36.198048 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0804964a-c388-4112-93b9-3c107b02a7c2 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:54:36.216138 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 654/1314] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 12:54:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 12:54:38.906559 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048dbf45-c97a-418d-82a3-70c4e71c2b9c None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:54:38.907068 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048dbf45-c97a-418d-82a3-70c4e71c2b9c None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:54:38.907163 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048dbf45-c97a-418d-82a3-70c4e71c2b9c None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:54:38.909207 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 661/1315] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:54:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:54:38.914673 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c8d44f-6fe9-46e3-a2b1-31f0fb5fb7b6 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:54:38.915065 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c8d44f-6fe9-46e3-a2b1-31f0fb5fb7b6 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:54:38.915307 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c8d44f-6fe9-46e3-a2b1-31f0fb5fb7b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:54:38.979263 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-65c8d44f-6fe9-46e3-a2b1-31f0fb5fb7b6 None None] MFA Rules not processed for user `281b2f886b53468d808994f33b5f60ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:54:38.980512 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-65c8d44f-6fe9-46e3-a2b1-31f0fb5fb7b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:54:39.033205 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:54:39.077560 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 655/1316] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:54:38 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 12:54:39.131977 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39554820-f04f-45d5-8766-16888e996839 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:54:39.132267 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39554820-f04f-45d5-8766-16888e996839 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:54:39.132546 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39554820-f04f-45d5-8766-16888e996839 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:54:39.133904 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 662/1317] 10.222.0.27 () {54 vars in 964 bytes} [Fri Jun 10 12:54:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:54:39.139724 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649ae884-feab-4b35-9752-0e9e543ed9b1 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:54:39.140048 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649ae884-feab-4b35-9752-0e9e543ed9b1 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:54:39.140326 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649ae884-feab-4b35-9752-0e9e543ed9b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:54:39.213659 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-649ae884-feab-4b35-9752-0e9e543ed9b1 None None] MFA Rules not processed for user `bc4415461a974c24ad7105a97c7d915b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:54:39.214834 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-649ae884-feab-4b35-9752-0e9e543ed9b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:54:39.287608 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 656/1318] 10.222.0.27 () {58 vars in 1061 bytes} [Fri Jun 10 12:54:39 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 12:54:39.296748 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39554820-f04f-45d5-8766-16888e996839 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:54:39.298265 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-39554820-f04f-45d5-8766-16888e996839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:54:39.307735 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:54:39.398281 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39554820-f04f-45d5-8766-16888e996839 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:54:39.470968 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5652efdf-57c9-430b-8766-02565855d87f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:54:39.471608 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5652efdf-57c9-430b-8766-02565855d87f service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:54:39.471908 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5652efdf-57c9-430b-8766-02565855d87f service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:54:39.472180 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5652efdf-57c9-430b-8766-02565855d87f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:54:39.473805 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5652efdf-57c9-430b-8766-02565855d87f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:54:39.533399 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:54:39.578542 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5652efdf-57c9-430b-8766-02565855d87f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:54:39.580515 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5652efdf-57c9-430b-8766-02565855d87f service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:54:39.598272 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 663/1319] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 12:54:39 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 12:54:42.920459 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74a5429-1148-417f-88c8-d97c10a1ad90 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:54:42.920981 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74a5429-1148-417f-88c8-d97c10a1ad90 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:54:42.921124 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74a5429-1148-417f-88c8-d97c10a1ad90 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:54:42.922501 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 657/1320] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:54:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:54:42.927653 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f351377-793f-4fe8-840a-16c0491a02ee None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:54:42.928112 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f351377-793f-4fe8-840a-16c0491a02ee None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:54:42.928254 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f351377-793f-4fe8-840a-16c0491a02ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:54:42.992536 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-1f351377-793f-4fe8-840a-16c0491a02ee None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:54:42.993744 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1f351377-793f-4fe8-840a-16c0491a02ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:54:43.088569 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 664/1321] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:54:42 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 12:54:43.126161 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74a5429-1148-417f-88c8-d97c10a1ad90 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:54:43.127695 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e74a5429-1148-417f-88c8-d97c10a1ad90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:54:43.177109 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:54:43.230979 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74a5429-1148-417f-88c8-d97c10a1ad90 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:54:43.302177 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd653751-1761-4c2a-af41-a2c8d71d42bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:54:43.302811 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd653751-1761-4c2a-af41-a2c8d71d42bc service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:54:43.303095 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd653751-1761-4c2a-af41-a2c8d71d42bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:54:43.303371 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd653751-1761-4c2a-af41-a2c8d71d42bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:54:43.305017 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-fd653751-1761-4c2a-af41-a2c8d71d42bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:54:43.398258 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:54:43.411253 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd653751-1761-4c2a-af41-a2c8d71d42bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:54:43.412713 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd653751-1761-4c2a-af41-a2c8d71d42bc service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:54:43.430329 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 658/1322] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 12:54:43 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 12:54:46.584391 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909ed2dd-2889-4901-ad97-6c663ae400e3 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:54:46.584918 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909ed2dd-2889-4901-ad97-6c663ae400e3 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:54:46.585009 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909ed2dd-2889-4901-ad97-6c663ae400e3 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:54:46.586479 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 665/1323] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:54:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:54:46.591581 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528538ec-bcff-489d-bb9e-834f0d593e2a None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:54:46.592036 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528538ec-bcff-489d-bb9e-834f0d593e2a None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:54:46.592171 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528538ec-bcff-489d-bb9e-834f0d593e2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:54:46.655559 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-528538ec-bcff-489d-bb9e-834f0d593e2a None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:54:46.656829 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-528538ec-bcff-489d-bb9e-834f0d593e2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:54:46.758109 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 659/1324] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:54:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 12:54:46.793838 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909ed2dd-2889-4901-ad97-6c663ae400e3 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:54:46.795386 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-909ed2dd-2889-4901-ad97-6c663ae400e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:54:46.816876 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:54:46.926678 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909ed2dd-2889-4901-ad97-6c663ae400e3 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:54:47.000694 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9222a9b3-d7f3-486d-8742-84770e815c80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:54:47.001369 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9222a9b3-d7f3-486d-8742-84770e815c80 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:54:47.001653 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9222a9b3-d7f3-486d-8742-84770e815c80 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:54:47.001927 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9222a9b3-d7f3-486d-8742-84770e815c80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:54:47.003548 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9222a9b3-d7f3-486d-8742-84770e815c80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:54:47.074322 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:54:47.112032 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9222a9b3-d7f3-486d-8742-84770e815c80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:54:47.113418 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9222a9b3-d7f3-486d-8742-84770e815c80 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:54:47.131002 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 666/1325] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 12:54:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 12:54:49.302744 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0342e8e7-d883-4335-88bb-f14cf373c02f None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:54:49.303270 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0342e8e7-d883-4335-88bb-f14cf373c02f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:54:49.303270 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0342e8e7-d883-4335-88bb-f14cf373c02f None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:54:49.304605 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 660/1326] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:54:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:54:49.309738 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d68be6-5144-423c-a8ab-339d1699a08f None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:54:49.310162 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d68be6-5144-423c-a8ab-339d1699a08f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:54:49.310289 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d68be6-5144-423c-a8ab-339d1699a08f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:54:49.373141 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-d7d68be6-5144-423c-a8ab-339d1699a08f None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:54:49.374462 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d7d68be6-5144-423c-a8ab-339d1699a08f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:54:49.469954 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 667/1327] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:54:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 12:54:49.505113 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0342e8e7-d883-4335-88bb-f14cf373c02f None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:54:49.506621 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0342e8e7-d883-4335-88bb-f14cf373c02f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:54:49.566568 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:54:49.610166 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0342e8e7-d883-4335-88bb-f14cf373c02f None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:54:49.686183 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8ebd58-63d9-406d-a5c2-6a14fc43de7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:54:49.686829 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8ebd58-63d9-406d-a5c2-6a14fc43de7b service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:54:49.687113 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8ebd58-63d9-406d-a5c2-6a14fc43de7b service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:54:49.687390 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8ebd58-63d9-406d-a5c2-6a14fc43de7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:54:49.689086 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-ea8ebd58-63d9-406d-a5c2-6a14fc43de7b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:54:49.795459 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:54:49.797914 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8ebd58-63d9-406d-a5c2-6a14fc43de7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:54:49.799356 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8ebd58-63d9-406d-a5c2-6a14fc43de7b service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:54:49.817288 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 661/1328] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 12:54:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 12:54:52.012226 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d83c20-7fd5-4f5e-8a0c-f5a27a48787f None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:54:52.012771 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d83c20-7fd5-4f5e-8a0c-f5a27a48787f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:54:52.012771 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d83c20-7fd5-4f5e-8a0c-f5a27a48787f None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:54:52.014273 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 668/1329] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:54:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:54:52.019327 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1d6252-b4a7-418d-b35a-71d9d312aac5 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:54:52.019729 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1d6252-b4a7-418d-b35a-71d9d312aac5 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:54:52.020044 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1d6252-b4a7-418d-b35a-71d9d312aac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:54:52.083969 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-4d1d6252-b4a7-418d-b35a-71d9d312aac5 None None] MFA Rules not processed for user `281b2f886b53468d808994f33b5f60ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:54:52.085178 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-4d1d6252-b4a7-418d-b35a-71d9d312aac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:54:52.179562 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 662/1330] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:54:52 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 12:54:52.216388 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d83c20-7fd5-4f5e-8a0c-f5a27a48787f None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:54:52.218012 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-22d83c20-7fd5-4f5e-8a0c-f5a27a48787f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:54:52.249025 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:54:52.320798 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d83c20-7fd5-4f5e-8a0c-f5a27a48787f None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:54:52.411502 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff733d9-66d9-445d-8682-9ee50cec2dc1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:54:52.412189 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff733d9-66d9-445d-8682-9ee50cec2dc1 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:54:52.412470 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff733d9-66d9-445d-8682-9ee50cec2dc1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:54:52.412743 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff733d9-66d9-445d-8682-9ee50cec2dc1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:54:52.414394 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3ff733d9-66d9-445d-8682-9ee50cec2dc1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:54:52.494132 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:54:52.523746 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff733d9-66d9-445d-8682-9ee50cec2dc1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:54:52.525199 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff733d9-66d9-445d-8682-9ee50cec2dc1 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:54:52.543497 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 669/1331] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 12:54:52 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 12:54:55.157464 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342ce9cc-ea99-4e61-ba52-1d91e3fc3f8d None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:54:55.158008 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342ce9cc-ea99-4e61-ba52-1d91e3fc3f8d None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:54:55.158008 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342ce9cc-ea99-4e61-ba52-1d91e3fc3f8d None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:54:55.159288 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 663/1332] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:54:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:54:55.164681 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8241d3d7-e8ed-4588-8bcf-bead885346f3 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:54:55.165097 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8241d3d7-e8ed-4588-8bcf-bead885346f3 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:54:55.165245 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8241d3d7-e8ed-4588-8bcf-bead885346f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:54:55.228378 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-8241d3d7-e8ed-4588-8bcf-bead885346f3 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:54:55.229564 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8241d3d7-e8ed-4588-8bcf-bead885346f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:54:55.323246 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 670/1333] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:54:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 10 12:54:55.358721 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342ce9cc-ea99-4e61-ba52-1d91e3fc3f8d None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:54:55.360285 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-342ce9cc-ea99-4e61-ba52-1d91e3fc3f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:54:55.429162 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:54:55.463371 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342ce9cc-ea99-4e61-ba52-1d91e3fc3f8d None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:54:55.533254 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755e420f-f4c6-4df2-ade7-cc3ccebf8b6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:54:55.533884 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755e420f-f4c6-4df2-ade7-cc3ccebf8b6b service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:54:55.534163 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755e420f-f4c6-4df2-ade7-cc3ccebf8b6b service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:54:55.534442 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755e420f-f4c6-4df2-ade7-cc3ccebf8b6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:54:55.536378 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-755e420f-f4c6-4df2-ade7-cc3ccebf8b6b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:54:55.642005 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755e420f-f4c6-4df2-ade7-cc3ccebf8b6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:54:55.643434 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755e420f-f4c6-4df2-ade7-cc3ccebf8b6b service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:54:55.653194 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:54:55.661244 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 664/1334] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 12:54:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 12:54:57.835565 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98e6f6c-4006-4e2c-9bec-c2dea5596a77 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:54:57.836205 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98e6f6c-4006-4e2c-9bec-c2dea5596a77 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:54:57.836205 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98e6f6c-4006-4e2c-9bec-c2dea5596a77 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:54:57.837576 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 671/1335] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:54:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:54:57.842622 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e2cd50-dd78-4e10-bed1-b0a4df21b88b None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:54:57.842997 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e2cd50-dd78-4e10-bed1-b0a4df21b88b None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:54:57.843173 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e2cd50-dd78-4e10-bed1-b0a4df21b88b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:54:57.906568 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-a4e2cd50-dd78-4e10-bed1-b0a4df21b88b None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:54:57.907766 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a4e2cd50-dd78-4e10-bed1-b0a4df21b88b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:54:58.003124 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 665/1336] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:54:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 12:54:58.042833 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98e6f6c-4006-4e2c-9bec-c2dea5596a77 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:54:58.044400 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e98e6f6c-4006-4e2c-9bec-c2dea5596a77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:54:58.084497 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:54:58.147395 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98e6f6c-4006-4e2c-9bec-c2dea5596a77 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:54:58.223044 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5121bcc2-2b53-46c4-96e2-d6f766ddf0d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:54:58.223691 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5121bcc2-2b53-46c4-96e2-d6f766ddf0d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:54:58.224005 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5121bcc2-2b53-46c4-96e2-d6f766ddf0d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:54:58.224283 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5121bcc2-2b53-46c4-96e2-d6f766ddf0d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:54:58.225908 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5121bcc2-2b53-46c4-96e2-d6f766ddf0d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:54:58.306724 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:54:58.331363 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5121bcc2-2b53-46c4-96e2-d6f766ddf0d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:54:58.332827 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5121bcc2-2b53-46c4-96e2-d6f766ddf0d2 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:54:58.352163 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 672/1337] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 12:54:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 12:55:01.082376 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7a93aa-ca19-4634-8f1f-033ef4d390f3 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:55:01.082958 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7a93aa-ca19-4634-8f1f-033ef4d390f3 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:55:01.082958 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7a93aa-ca19-4634-8f1f-033ef4d390f3 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:55:01.084317 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 666/1338] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:55:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:55:01.089390 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa0fc8a-c3b5-42f7-98e8-3d7504248067 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:55:01.089788 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa0fc8a-c3b5-42f7-98e8-3d7504248067 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:55:01.089959 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa0fc8a-c3b5-42f7-98e8-3d7504248067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:55:01.153657 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-8fa0fc8a-c3b5-42f7-98e8-3d7504248067 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:55:01.154800 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8fa0fc8a-c3b5-42f7-98e8-3d7504248067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:55:01.250834 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 673/1339] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:55:01 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 12:55:01.287139 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7a93aa-ca19-4634-8f1f-033ef4d390f3 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:55:01.288723 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-db7a93aa-ca19-4634-8f1f-033ef4d390f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:55:01.362654 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:55:01.389812 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7a93aa-ca19-4634-8f1f-033ef4d390f3 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:55:01.461703 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb44ceb6-3b2e-4fb4-a8f5-7e2b3fbeea93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:55:01.462269 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb44ceb6-3b2e-4fb4-a8f5-7e2b3fbeea93 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:55:01.462549 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb44ceb6-3b2e-4fb4-a8f5-7e2b3fbeea93 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:55:01.462820 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb44ceb6-3b2e-4fb4-a8f5-7e2b3fbeea93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:55:01.464467 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-fb44ceb6-3b2e-4fb4-a8f5-7e2b3fbeea93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:55:01.569288 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb44ceb6-3b2e-4fb4-a8f5-7e2b3fbeea93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:55:01.570698 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb44ceb6-3b2e-4fb4-a8f5-7e2b3fbeea93 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:55:01.588467 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 667/1340] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 12:55:01 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 12:55:05.976146 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a661e6-a5b3-405b-8f68-a949026e802c None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:55:05.976726 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a661e6-a5b3-405b-8f68-a949026e802c None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:55:05.976726 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a661e6-a5b3-405b-8f68-a949026e802c None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:55:05.978147 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 674/1341] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:55:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:55:05.983501 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2228f609-9b77-411e-be2c-f2088e67a6fa None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:55:05.983958 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2228f609-9b77-411e-be2c-f2088e67a6fa None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:55:05.984164 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2228f609-9b77-411e-be2c-f2088e67a6fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:55:05.990372 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:55:06.057536 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-2228f609-9b77-411e-be2c-f2088e67a6fa None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:55:06.058740 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-2228f609-9b77-411e-be2c-f2088e67a6fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:55:06.164677 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 668/1342] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:55:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 12:55:06.203709 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a661e6-a5b3-405b-8f68-a949026e802c None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:55:06.206118 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-03a661e6-a5b3-405b-8f68-a949026e802c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:55:06.256103 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:55:06.301611 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a661e6-a5b3-405b-8f68-a949026e802c None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:55:06.305323 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d3d000-965a-418b-8555-dc2ca8cdc665 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:55:06.306076 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d3d000-965a-418b-8555-dc2ca8cdc665 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:55:06.306437 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d3d000-965a-418b-8555-dc2ca8cdc665 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:55:06.306779 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d3d000-965a-418b-8555-dc2ca8cdc665 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:55:06.316521 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d3d000-965a-418b-8555-dc2ca8cdc665 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:55:06.318018 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d3d000-965a-418b-8555-dc2ca8cdc665 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:55:06.327437 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 675/1343] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 12:55:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 12:55:09.364649 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07359066-d37b-4184-b68e-9909641d2470 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:55:09.365178 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07359066-d37b-4184-b68e-9909641d2470 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:55:09.365178 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07359066-d37b-4184-b68e-9909641d2470 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:55:09.366560 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 669/1344] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:55:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:55:09.371710 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed2e658-14c3-444a-93ab-9d532531d1c9 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:55:09.372121 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed2e658-14c3-444a-93ab-9d532531d1c9 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:55:09.372448 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed2e658-14c3-444a-93ab-9d532531d1c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:55:09.400332 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-7ed2e658-14c3-444a-93ab-9d532531d1c9 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:55:09.401539 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-7ed2e658-14c3-444a-93ab-9d532531d1c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:55:09.478726 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 676/1345] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:55:09 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 12:55:09.515173 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07359066-d37b-4184-b68e-9909641d2470 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:55:09.525896 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07359066-d37b-4184-b68e-9909641d2470 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:55:09.529523 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c434c6d-d64b-4dfe-9955-5c4b23b95d74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:55:09.530214 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c434c6d-d64b-4dfe-9955-5c4b23b95d74 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:55:09.530583 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c434c6d-d64b-4dfe-9955-5c4b23b95d74 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:55:09.530926 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c434c6d-d64b-4dfe-9955-5c4b23b95d74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:55:09.540893 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c434c6d-d64b-4dfe-9955-5c4b23b95d74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:55:09.542348 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c434c6d-d64b-4dfe-9955-5c4b23b95d74 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:55:09.554163 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 670/1346] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 12:55:09 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 12:55:13.943038 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26909ae-0c7b-4f6c-85af-d2721292fc5b None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:55:13.943589 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26909ae-0c7b-4f6c-85af-d2721292fc5b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:55:13.943589 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26909ae-0c7b-4f6c-85af-d2721292fc5b None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:55:13.944997 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 677/1347] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:55:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:55:13.950227 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7725073-c9aa-432d-a6a7-aa63776d0e3b None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:55:13.950678 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7725073-c9aa-432d-a6a7-aa63776d0e3b None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:55:13.950815 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7725073-c9aa-432d-a6a7-aa63776d0e3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:55:13.969646 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-a7725073-c9aa-432d-a6a7-aa63776d0e3b None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:55:13.970879 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a7725073-c9aa-432d-a6a7-aa63776d0e3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:55:13.978874 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 671/1348] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:55:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 12:55:14.017894 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26909ae-0c7b-4f6c-85af-d2721292fc5b None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:55:14.028392 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26909ae-0c7b-4f6c-85af-d2721292fc5b None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:55:14.031817 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf9107e-6aec-4a83-9157-c5c63eb1dd7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:55:14.032478 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf9107e-6aec-4a83-9157-c5c63eb1dd7b service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:55:14.032756 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf9107e-6aec-4a83-9157-c5c63eb1dd7b service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:55:14.033030 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf9107e-6aec-4a83-9157-c5c63eb1dd7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:55:14.041843 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf9107e-6aec-4a83-9157-c5c63eb1dd7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:55:14.043241 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf9107e-6aec-4a83-9157-c5c63eb1dd7b service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:55:14.056106 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 678/1349] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 12:55:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 12:55:19.072101 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c1d9f2-5c35-4297-ad39-dd49a3e2f909 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:55:19.072975 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c1d9f2-5c35-4297-ad39-dd49a3e2f909 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:55:19.073505 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c1d9f2-5c35-4297-ad39-dd49a3e2f909 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:55:19.075629 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 672/1350] 10.222.0.27 () {54 vars in 921 bytes} [Fri Jun 10 12:55:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:55:19.081129 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df7c50e-d88c-40f3-942a-1728c424fb3c None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:55:19.081902 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df7c50e-d88c-40f3-942a-1728c424fb3c None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:55:19.082355 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df7c50e-d88c-40f3-942a-1728c424fb3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:55:19.158037 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-6df7c50e-d88c-40f3-942a-1728c424fb3c None None] MFA Rules not processed for user `d972ab782b3f4d2aba575314f16dafc7`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:55:19.159912 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6df7c50e-d88c-40f3-942a-1728c424fb3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:55:19.190682 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:55:19.238565 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 679/1351] 10.222.0.27 () {58 vars in 1018 bytes} [Fri Jun 10 12:55:19 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 12:55:23.475529 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54e50a8-2e71-45cd-9be3-86f48a549846 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:55:23.476105 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54e50a8-2e71-45cd-9be3-86f48a549846 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:55:23.476190 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54e50a8-2e71-45cd-9be3-86f48a549846 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:55:23.477505 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 673/1352] 10.222.0.27 () {54 vars in 921 bytes} [Fri Jun 10 12:55:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:55:23.482821 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60b93b5-d8ff-4d08-8213-e93795686cad None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:55:23.483736 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60b93b5-d8ff-4d08-8213-e93795686cad None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:55:23.484280 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60b93b5-d8ff-4d08-8213-e93795686cad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:55:23.503931 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-f60b93b5-d8ff-4d08-8213-e93795686cad None None] MFA Rules not processed for user `d972ab782b3f4d2aba575314f16dafc7`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:55:23.505125 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f60b93b5-d8ff-4d08-8213-e93795686cad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:55:23.513797 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 680/1353] 10.222.0.27 () {58 vars in 1018 bytes} [Fri Jun 10 12:55:23 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 12:55:23.972314 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=72 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:55:24.999339 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec886b7f-7b77-4ad7-8a9a-291c5662063b None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:55:25.000664 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec886b7f-7b77-4ad7-8a9a-291c5662063b None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:55:25.000664 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec886b7f-7b77-4ad7-8a9a-291c5662063b None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:55:25.002208 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 674/1354] 10.222.0.27 () {54 vars in 921 bytes} [Fri Jun 10 12:55:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:55:25.007442 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18217453-aac1-4044-83e1-61fe8d02b361 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:55:25.007897 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18217453-aac1-4044-83e1-61fe8d02b361 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:55:25.008052 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18217453-aac1-4044-83e1-61fe8d02b361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:55:25.026922 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-18217453-aac1-4044-83e1-61fe8d02b361 None None] MFA Rules not processed for user `d972ab782b3f4d2aba575314f16dafc7`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:55:25.028143 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-18217453-aac1-4044-83e1-61fe8d02b361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:55:25.037062 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 681/1355] 10.222.0.27 () {58 vars in 1018 bytes} [Fri Jun 10 12:55:25 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 10 12:55:28.733407 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f72e241-85b8-49af-bef4-3856c6df3f5a None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:55:28.733975 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f72e241-85b8-49af-bef4-3856c6df3f5a None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:55:28.733975 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f72e241-85b8-49af-bef4-3856c6df3f5a None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:55:28.735330 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 675/1356] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:55:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:55:28.740454 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d90bd0-e60e-49e3-9f7c-8f60086ed3cf None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:55:28.740864 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d90bd0-e60e-49e3-9f7c-8f60086ed3cf None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:55:28.741014 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d90bd0-e60e-49e3-9f7c-8f60086ed3cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:55:28.758703 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-d7d90bd0-e60e-49e3-9f7c-8f60086ed3cf None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:55:28.759891 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d7d90bd0-e60e-49e3-9f7c-8f60086ed3cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:55:28.768228 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 682/1357] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:55:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 12:55:28.820905 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f37f2f-831d-4cc3-a819-13d97991e8ca None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:55:28.829063 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f37f2f-831d-4cc3-a819-13d97991e8ca None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:55:28.829063 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f37f2f-831d-4cc3-a819-13d97991e8ca None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:55:28.829063 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 676/1358] 10.222.0.27 () {54 vars in 961 bytes} [Fri Jun 10 12:55:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:55:28.840335 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21c010d-c366-4325-b3d3-cee9474af317 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:55:28.840710 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21c010d-c366-4325-b3d3-cee9474af317 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:55:28.841211 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21c010d-c366-4325-b3d3-cee9474af317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:55:28.899950 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-d21c010d-c366-4325-b3d3-cee9474af317 None None] MFA Rules not processed for user `8bd6caa461e84100a3d5fe5af5d80f3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:55:28.901515 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d21c010d-c366-4325-b3d3-cee9474af317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:55:28.990628 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 683/1359] 10.222.0.27 () {58 vars in 1058 bytes} [Fri Jun 10 12:55:28 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 10 12:55:28.998892 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f37f2f-831d-4cc3-a819-13d97991e8ca None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:55:29.014328 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f37f2f-831d-4cc3-a819-13d97991e8ca None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:55:29.020464 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe3bb1b-d7fd-4060-bd48-f88b5aa37f9e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bd6caa461e84100a3d5fe5af5d80f3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:55:29.021280 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe3bb1b-d7fd-4060-bd48-f88b5aa37f9e service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:55:29.021637 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe3bb1b-d7fd-4060-bd48-f88b5aa37f9e service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:55:29.022147 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe3bb1b-d7fd-4060-bd48-f88b5aa37f9e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:55:29.031697 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe3bb1b-d7fd-4060-bd48-f88b5aa37f9e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:55:29.033517 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe3bb1b-d7fd-4060-bd48-f88b5aa37f9e service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:55:29.044083 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 677/1360] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 12:55:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 12:55:31.369234 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde7ab5c-037c-4dbd-83db-4265099c3ef7 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:55:31.369811 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde7ab5c-037c-4dbd-83db-4265099c3ef7 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:55:31.369811 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde7ab5c-037c-4dbd-83db-4265099c3ef7 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:55:31.371319 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 684/1361] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:55:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:55:31.376271 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fc2f09-bd08-4750-91b4-13c94ef023c4 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:55:31.376704 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fc2f09-bd08-4750-91b4-13c94ef023c4 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:55:31.376820 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fc2f09-bd08-4750-91b4-13c94ef023c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:55:31.396681 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-44fc2f09-bd08-4750-91b4-13c94ef023c4 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:55:31.398310 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-44fc2f09-bd08-4750-91b4-13c94ef023c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:55:31.408183 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 678/1362] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:55:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 12:55:31.454969 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde7ab5c-037c-4dbd-83db-4265099c3ef7 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:55:31.471015 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde7ab5c-037c-4dbd-83db-4265099c3ef7 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:55:31.476831 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2cb1f6-aac4-4cbd-bffd-450a925ff423 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bd6caa461e84100a3d5fe5af5d80f3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:55:31.477593 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2cb1f6-aac4-4cbd-bffd-450a925ff423 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:55:31.477947 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2cb1f6-aac4-4cbd-bffd-450a925ff423 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:55:31.478295 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2cb1f6-aac4-4cbd-bffd-450a925ff423 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:55:31.488950 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2cb1f6-aac4-4cbd-bffd-450a925ff423 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:55:31.490620 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2cb1f6-aac4-4cbd-bffd-450a925ff423 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:55:31.502421 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 685/1363] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 12:55:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 12:55:34.126598 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d1100a-1139-4c79-b024-cc2620f0ca02 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:55:34.127096 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d1100a-1139-4c79-b024-cc2620f0ca02 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:55:34.127210 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d1100a-1139-4c79-b024-cc2620f0ca02 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:55:34.128616 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 679/1364] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:55:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:55:34.133745 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27861e7a-bc01-4cb1-8339-d0b7fcc20954 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:55:34.134187 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27861e7a-bc01-4cb1-8339-d0b7fcc20954 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:55:34.134310 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27861e7a-bc01-4cb1-8339-d0b7fcc20954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:55:34.152098 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-27861e7a-bc01-4cb1-8339-d0b7fcc20954 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:55:34.153291 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-27861e7a-bc01-4cb1-8339-d0b7fcc20954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:55:34.161265 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 686/1365] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:55:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 12:55:34.208494 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4195c73b-5bc4-4e6a-a835-06e9826501ca None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:55:34.208784 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4195c73b-5bc4-4e6a-a835-06e9826501ca None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:55:34.209064 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4195c73b-5bc4-4e6a-a835-06e9826501ca None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:55:34.210442 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 680/1366] 10.222.0.27 () {54 vars in 961 bytes} [Fri Jun 10 12:55:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:55:34.216201 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099a1c6e-9ff9-45ef-9277-0bc1bea0a796 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:55:34.216488 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099a1c6e-9ff9-45ef-9277-0bc1bea0a796 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:55:34.216759 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099a1c6e-9ff9-45ef-9277-0bc1bea0a796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:55:34.237593 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-099a1c6e-9ff9-45ef-9277-0bc1bea0a796 None None] MFA Rules not processed for user `8bd6caa461e84100a3d5fe5af5d80f3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:55:34.238769 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-099a1c6e-9ff9-45ef-9277-0bc1bea0a796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:55:34.249140 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 687/1367] 10.222.0.27 () {58 vars in 1058 bytes} [Fri Jun 10 12:55:34 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 10 12:55:34.257413 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4195c73b-5bc4-4e6a-a835-06e9826501ca None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:55:34.278124 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4195c73b-5bc4-4e6a-a835-06e9826501ca None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:55:34.286760 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab2f80b-49d3-4932-acfd-29f1425ea9a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bd6caa461e84100a3d5fe5af5d80f3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:55:34.287739 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab2f80b-49d3-4932-acfd-29f1425ea9a2 service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:55:34.288269 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab2f80b-49d3-4932-acfd-29f1425ea9a2 service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:55:34.288767 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab2f80b-49d3-4932-acfd-29f1425ea9a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:55:34.299052 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab2f80b-49d3-4932-acfd-29f1425ea9a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:55:34.300678 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab2f80b-49d3-4932-acfd-29f1425ea9a2 service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:55:34.327922 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 681/1368] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 12:55:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 12:55:36.651353 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151438ac-107f-4bc1-b9f0-cb7cbc660a26 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:55:36.651916 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151438ac-107f-4bc1-b9f0-cb7cbc660a26 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:55:36.652349 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151438ac-107f-4bc1-b9f0-cb7cbc660a26 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:55:36.654054 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 688/1369] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:55:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:55:36.659421 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab87b530-3c36-4d8b-8e40-a90b7ea6d613 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:55:36.659914 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab87b530-3c36-4d8b-8e40-a90b7ea6d613 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:55:36.660001 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab87b530-3c36-4d8b-8e40-a90b7ea6d613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:55:36.678295 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-ab87b530-3c36-4d8b-8e40-a90b7ea6d613 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:55:36.679477 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-ab87b530-3c36-4d8b-8e40-a90b7ea6d613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:55:36.687907 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 682/1370] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:55:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 12:55:36.731117 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151438ac-107f-4bc1-b9f0-cb7cbc660a26 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:55:36.746114 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151438ac-107f-4bc1-b9f0-cb7cbc660a26 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:55:36.752241 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d323a92-19b9-4411-8025-4d0ff5241fca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bd6caa461e84100a3d5fe5af5d80f3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:55:36.752991 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d323a92-19b9-4411-8025-4d0ff5241fca service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:55:36.753372 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d323a92-19b9-4411-8025-4d0ff5241fca service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:55:36.753748 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d323a92-19b9-4411-8025-4d0ff5241fca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:55:36.762622 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d323a92-19b9-4411-8025-4d0ff5241fca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:55:36.764348 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d323a92-19b9-4411-8025-4d0ff5241fca service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:55:36.775148 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 689/1371] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 12:55:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 10 12:55:36.949749 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d5a41c-4a01-4bbb-9857-e20d12119a65 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:55:36.949989 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d5a41c-4a01-4bbb-9857-e20d12119a65 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:55:36.950255 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d5a41c-4a01-4bbb-9857-e20d12119a65 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:55:36.951650 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 683/1372] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:55:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:55:36.956995 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfc45c4-94ed-4b7f-98fd-05ced271d16e None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:55:36.957372 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfc45c4-94ed-4b7f-98fd-05ced271d16e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:55:36.957758 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfc45c4-94ed-4b7f-98fd-05ced271d16e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:55:36.977162 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-7dfc45c4-94ed-4b7f-98fd-05ced271d16e None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:55:36.978490 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-7dfc45c4-94ed-4b7f-98fd-05ced271d16e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:55:36.991907 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 690/1373] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:55:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 12:55:37.297724 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4d8ea3-4fdd-4a24-bb3c-39ea0c0d94f3 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:55:37.298121 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4d8ea3-4fdd-4a24-bb3c-39ea0c0d94f3 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:55:37.298589 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4d8ea3-4fdd-4a24-bb3c-39ea0c0d94f3 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:55:37.300376 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 684/1374] 10.222.0.27 () {54 vars in 963 bytes} [Fri Jun 10 12:55:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:55:37.305298 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928ac5ba-0274-4f58-afa3-9dc3ab6a2f9e None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:55:37.305692 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928ac5ba-0274-4f58-afa3-9dc3ab6a2f9e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:55:37.306073 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928ac5ba-0274-4f58-afa3-9dc3ab6a2f9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:55:37.312929 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:55:37.358115 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-928ac5ba-0274-4f58-afa3-9dc3ab6a2f9e None None] MFA Rules not processed for user `c6cc95a563b941ac92ab283aa0d5a5d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:55:37.360083 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-928ac5ba-0274-4f58-afa3-9dc3ab6a2f9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:55:37.435546 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 691/1375] 10.222.0.27 () {58 vars in 1060 bytes} [Fri Jun 10 12:55:37 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 10 12:55:37.443504 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4d8ea3-4fdd-4a24-bb3c-39ea0c0d94f3 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:55:37.454945 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4d8ea3-4fdd-4a24-bb3c-39ea0c0d94f3 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:55:37.459527 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11116e66-825e-471b-9804-8fbd69367382 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6cc95a563b941ac92ab283aa0d5a5d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:55:37.460194 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11116e66-825e-471b-9804-8fbd69367382 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:55:37.460491 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11116e66-825e-471b-9804-8fbd69367382 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:55:37.460750 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11116e66-825e-471b-9804-8fbd69367382 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:55:37.471059 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11116e66-825e-471b-9804-8fbd69367382 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:55:37.472518 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11116e66-825e-471b-9804-8fbd69367382 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:55:37.482231 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 685/1376] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 12:55:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 12:55:39.070216 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb701f91-fbfc-4ba9-a62a-0e3e2af61879 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:55:39.070752 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb701f91-fbfc-4ba9-a62a-0e3e2af61879 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:55:39.070752 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb701f91-fbfc-4ba9-a62a-0e3e2af61879 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:55:39.072220 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 692/1377] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:55:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:55:39.078479 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03990f2-0abb-447a-81b6-652d75b5af9a None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:55:39.078881 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03990f2-0abb-447a-81b6-652d75b5af9a None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:55:39.079022 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03990f2-0abb-447a-81b6-652d75b5af9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:55:39.097324 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-b03990f2-0abb-447a-81b6-652d75b5af9a None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:55:39.098521 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b03990f2-0abb-447a-81b6-652d75b5af9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:55:39.106472 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 686/1378] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:55:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 12:55:39.151670 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb701f91-fbfc-4ba9-a62a-0e3e2af61879 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:55:39.167407 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb701f91-fbfc-4ba9-a62a-0e3e2af61879 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:55:39.173108 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7150b2a-f78e-4f67-87d4-62d15ff3b2f1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bd6caa461e84100a3d5fe5af5d80f3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:55:39.175930 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7150b2a-f78e-4f67-87d4-62d15ff3b2f1 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:55:39.175930 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7150b2a-f78e-4f67-87d4-62d15ff3b2f1 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:55:39.175930 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7150b2a-f78e-4f67-87d4-62d15ff3b2f1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:55:39.183627 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7150b2a-f78e-4f67-87d4-62d15ff3b2f1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:55:39.185327 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7150b2a-f78e-4f67-87d4-62d15ff3b2f1 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:55:39.196155 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 693/1379] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 12:55:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 12:55:41.552537 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e12e1b9-ba13-45db-ae8f-dda4ae366a63 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:55:41.553114 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e12e1b9-ba13-45db-ae8f-dda4ae366a63 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:55:41.553114 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e12e1b9-ba13-45db-ae8f-dda4ae366a63 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:55:41.554472 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 687/1380] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:55:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:55:41.559583 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245f50ce-6b9a-40c4-af2f-498305cb61ee None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:55:41.560057 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245f50ce-6b9a-40c4-af2f-498305cb61ee None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:55:41.560157 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245f50ce-6b9a-40c4-af2f-498305cb61ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:55:41.578604 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-245f50ce-6b9a-40c4-af2f-498305cb61ee None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:55:41.579794 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-245f50ce-6b9a-40c4-af2f-498305cb61ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:55:41.587749 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 694/1381] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:55:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 12:55:41.630750 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e12e1b9-ba13-45db-ae8f-dda4ae366a63 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:55:41.647351 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e12e1b9-ba13-45db-ae8f-dda4ae366a63 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:55:41.652978 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43133a7e-c13a-4b05-9f6b-b8d63cd04aa6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bd6caa461e84100a3d5fe5af5d80f3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:55:41.653652 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43133a7e-c13a-4b05-9f6b-b8d63cd04aa6 service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:55:41.654111 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43133a7e-c13a-4b05-9f6b-b8d63cd04aa6 service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:55:41.654552 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43133a7e-c13a-4b05-9f6b-b8d63cd04aa6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:55:41.664865 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43133a7e-c13a-4b05-9f6b-b8d63cd04aa6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:55:41.666957 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43133a7e-c13a-4b05-9f6b-b8d63cd04aa6 service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:55:41.678353 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 688/1382] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 12:55:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 12:55:43.982049 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed9cf1c-68b1-43de-b4be-22c8834ad2ac None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:55:43.982571 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed9cf1c-68b1-43de-b4be-22c8834ad2ac None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:55:43.982571 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed9cf1c-68b1-43de-b4be-22c8834ad2ac None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:55:43.984493 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 695/1383] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:55:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:55:43.989895 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba5572d-b69e-4b95-a678-01e03a2037b3 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:55:43.990328 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba5572d-b69e-4b95-a678-01e03a2037b3 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:55:43.990451 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba5572d-b69e-4b95-a678-01e03a2037b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:55:44.008259 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-dba5572d-b69e-4b95-a678-01e03a2037b3 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:55:44.009434 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-dba5572d-b69e-4b95-a678-01e03a2037b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:55:44.017460 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 689/1384] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:55:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 12:55:44.060490 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed9cf1c-68b1-43de-b4be-22c8834ad2ac None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:55:44.076044 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed9cf1c-68b1-43de-b4be-22c8834ad2ac None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:55:44.082405 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971f7970-f8ed-46f9-ae7c-c62f34d771f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bd6caa461e84100a3d5fe5af5d80f3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:55:44.083417 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971f7970-f8ed-46f9-ae7c-c62f34d771f8 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:55:44.083852 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971f7970-f8ed-46f9-ae7c-c62f34d771f8 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:55:44.084313 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971f7970-f8ed-46f9-ae7c-c62f34d771f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:55:44.094966 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971f7970-f8ed-46f9-ae7c-c62f34d771f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:55:44.096973 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971f7970-f8ed-46f9-ae7c-c62f34d771f8 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:55:44.109163 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 696/1385] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 12:55:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 12:55:46.419397 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e99a57-b077-4027-bb9b-19c852eb22d5 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:55:46.420305 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e99a57-b077-4027-bb9b-19c852eb22d5 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:55:46.420305 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e99a57-b077-4027-bb9b-19c852eb22d5 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:55:46.421940 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 690/1386] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:55:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:55:46.427305 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633384be-33d1-426e-a246-206a73ab859c None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:55:46.427839 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633384be-33d1-426e-a246-206a73ab859c None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:55:46.427839 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633384be-33d1-426e-a246-206a73ab859c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:55:46.448707 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-633384be-33d1-426e-a246-206a73ab859c None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:55:46.450605 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-633384be-33d1-426e-a246-206a73ab859c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:55:46.461314 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 697/1387] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:55:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 12:55:46.506936 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e99a57-b077-4027-bb9b-19c852eb22d5 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:55:46.521997 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e99a57-b077-4027-bb9b-19c852eb22d5 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:55:46.527692 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3e77d5-3720-49e2-b619-cbb16b65de30 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bd6caa461e84100a3d5fe5af5d80f3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:55:46.528438 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3e77d5-3720-49e2-b619-cbb16b65de30 service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:55:46.528727 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3e77d5-3720-49e2-b619-cbb16b65de30 service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:55:46.529009 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3e77d5-3720-49e2-b619-cbb16b65de30 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:55:46.538613 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3e77d5-3720-49e2-b619-cbb16b65de30 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:55:46.540250 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3e77d5-3720-49e2-b619-cbb16b65de30 service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:55:46.551326 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 691/1388] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 12:55:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 12:55:48.889145 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42ce446-6b56-442f-9235-de6760721d72 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:55:48.890236 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42ce446-6b56-442f-9235-de6760721d72 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:55:48.890553 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42ce446-6b56-442f-9235-de6760721d72 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:55:48.892580 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 698/1389] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:55:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:55:48.898138 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44774900-e225-4c0b-8a90-7c1c4dee1ebf None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:55:48.899600 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44774900-e225-4c0b-8a90-7c1c4dee1ebf None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:55:48.899600 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44774900-e225-4c0b-8a90-7c1c4dee1ebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:55:48.918927 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-44774900-e225-4c0b-8a90-7c1c4dee1ebf None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:55:48.920174 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-44774900-e225-4c0b-8a90-7c1c4dee1ebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:55:48.929707 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 692/1390] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:55:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 12:55:48.976600 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42ce446-6b56-442f-9235-de6760721d72 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:55:48.990910 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42ce446-6b56-442f-9235-de6760721d72 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:55:48.996787 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e996141-58b3-4a9a-ba94-6aced8794370 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bd6caa461e84100a3d5fe5af5d80f3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:55:48.997525 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e996141-58b3-4a9a-ba94-6aced8794370 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:55:48.997833 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e996141-58b3-4a9a-ba94-6aced8794370 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:55:48.998130 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e996141-58b3-4a9a-ba94-6aced8794370 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:55:49.007081 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e996141-58b3-4a9a-ba94-6aced8794370 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:55:49.008772 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e996141-58b3-4a9a-ba94-6aced8794370 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:55:49.019672 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 699/1391] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 12:55:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 12:55:51.444315 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59893634-f3ed-4986-b7f8-32f84a699948 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:55:51.444862 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59893634-f3ed-4986-b7f8-32f84a699948 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:55:51.444862 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59893634-f3ed-4986-b7f8-32f84a699948 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:55:51.446252 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 693/1392] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:55:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 10 12:55:51.451324 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc7095-dc8b-45d6-afd6-7c45560ee29f None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:55:51.451749 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc7095-dc8b-45d6-afd6-7c45560ee29f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:55:51.451885 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebc7095-dc8b-45d6-afd6-7c45560ee29f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:55:51.470074 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-9ebc7095-dc8b-45d6-afd6-7c45560ee29f None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:55:51.471276 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9ebc7095-dc8b-45d6-afd6-7c45560ee29f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:55:51.479106 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 700/1393] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:55:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 12:55:51.524176 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59893634-f3ed-4986-b7f8-32f84a699948 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:55:51.539033 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59893634-f3ed-4986-b7f8-32f84a699948 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:55:51.544415 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a923d2-d9c6-4753-bd8d-ace62bf5f34b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bd6caa461e84100a3d5fe5af5d80f3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:55:51.545059 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a923d2-d9c6-4753-bd8d-ace62bf5f34b service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:55:51.545341 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a923d2-d9c6-4753-bd8d-ace62bf5f34b service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:55:51.545617 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a923d2-d9c6-4753-bd8d-ace62bf5f34b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:55:51.554831 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a923d2-d9c6-4753-bd8d-ace62bf5f34b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:55:51.556452 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a923d2-d9c6-4753-bd8d-ace62bf5f34b service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:55:51.566029 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 694/1394] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 12:55:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 12:55:52.964727 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf71dda-7922-4c39-bb5a-497b843f133e None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:55:52.965299 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf71dda-7922-4c39-bb5a-497b843f133e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:55:52.965299 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf71dda-7922-4c39-bb5a-497b843f133e None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:55:52.966642 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 701/1395] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:55:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:55:52.971795 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ecf25e-b1c1-46e7-9245-769d931e4c4f None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:55:52.972221 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ecf25e-b1c1-46e7-9245-769d931e4c4f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:55:52.972396 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ecf25e-b1c1-46e7-9245-769d931e4c4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:55:52.989794 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-84ecf25e-b1c1-46e7-9245-769d931e4c4f None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:55:52.990980 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-84ecf25e-b1c1-46e7-9245-769d931e4c4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:55:52.998818 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 695/1396] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:55:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 12:55:53.053782 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82000a00-75cd-4b06-9074-0b4ef439ea80 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:55:53.054069 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82000a00-75cd-4b06-9074-0b4ef439ea80 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:55:53.054348 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82000a00-75cd-4b06-9074-0b4ef439ea80 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:55:53.055691 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 702/1397] 10.222.0.27 () {54 vars in 962 bytes} [Fri Jun 10 12:55:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:55:53.060738 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e811a6c0-2a17-4b44-8b00-a617c360327a None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:55:53.061029 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e811a6c0-2a17-4b44-8b00-a617c360327a None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:55:53.061308 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e811a6c0-2a17-4b44-8b00-a617c360327a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:55:53.110356 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-e811a6c0-2a17-4b44-8b00-a617c360327a None None] MFA Rules not processed for user `6d3eb01a98274050b930616b98998c06`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:55:53.111560 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e811a6c0-2a17-4b44-8b00-a617c360327a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:55:53.171613 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 696/1398] 10.222.0.27 () {58 vars in 1059 bytes} [Fri Jun 10 12:55:53 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 12:55:53.178949 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82000a00-75cd-4b06-9074-0b4ef439ea80 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:55:53.190162 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82000a00-75cd-4b06-9074-0b4ef439ea80 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:55:53.193728 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebfd9d8-40ba-4926-841e-56931eb43a23 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d3eb01a98274050b930616b98998c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:55:53.194362 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebfd9d8-40ba-4926-841e-56931eb43a23 service glance] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:55:53.194647 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebfd9d8-40ba-4926-841e-56931eb43a23 service glance] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:55:53.194929 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebfd9d8-40ba-4926-841e-56931eb43a23 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:55:53.204733 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebfd9d8-40ba-4926-841e-56931eb43a23 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:55:53.206160 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebfd9d8-40ba-4926-841e-56931eb43a23 service glance] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:55:53.215569 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 703/1399] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 12:55:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 12:55:53.850261 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bcb7c3-526e-4a75-8752-5d60645d712d None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:55:53.850483 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bcb7c3-526e-4a75-8752-5d60645d712d None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:55:53.850768 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bcb7c3-526e-4a75-8752-5d60645d712d None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:55:53.852361 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 697/1400] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:55:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:55:53.857724 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1409c8a5-f92d-48aa-8537-4a1aa5bf194a None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:55:53.858046 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1409c8a5-f92d-48aa-8537-4a1aa5bf194a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:55:53.858331 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1409c8a5-f92d-48aa-8537-4a1aa5bf194a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:55:53.878948 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-1409c8a5-f92d-48aa-8537-4a1aa5bf194a None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:55:53.880300 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1409c8a5-f92d-48aa-8537-4a1aa5bf194a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:55:53.889085 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 704/1401] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:55:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 12:55:53.933148 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bcb7c3-526e-4a75-8752-5d60645d712d None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:55:53.948198 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bcb7c3-526e-4a75-8752-5d60645d712d None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:55:53.954295 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faadeb4f-4590-4795-9db9-48de94c25498 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bd6caa461e84100a3d5fe5af5d80f3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:55:53.954940 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faadeb4f-4590-4795-9db9-48de94c25498 service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:55:53.959988 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faadeb4f-4590-4795-9db9-48de94c25498 service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:55:53.959988 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faadeb4f-4590-4795-9db9-48de94c25498 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:55:53.963936 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faadeb4f-4590-4795-9db9-48de94c25498 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:55:53.965518 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faadeb4f-4590-4795-9db9-48de94c25498 service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:55:53.974985 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 698/1402] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 12:55:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 12:55:54.145861 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21371fd-3d30-467f-9e1e-f520b5d7abd2 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:55:54.146275 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21371fd-3d30-467f-9e1e-f520b5d7abd2 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:55:54.146433 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21371fd-3d30-467f-9e1e-f520b5d7abd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:55:54.196941 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-f21371fd-3d30-467f-9e1e-f520b5d7abd2 None None] MFA Rules not processed for user `da66362070694e91bbe2cfbbd161b8a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:55:54.198269 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f21371fd-3d30-467f-9e1e-f520b5d7abd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:55:54.234989 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:55:54.269704 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 705/1403] 10.222.0.27 () {58 vars in 1009 bytes} [Fri Jun 10 12:55:54 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 12:55:54.314208 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f0bc88-5f92-4387-b28f-dca82885daae None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:55:54.314613 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f0bc88-5f92-4387-b28f-dca82885daae None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:55:54.314897 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f0bc88-5f92-4387-b28f-dca82885daae None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:55:54.316407 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 699/1404] 10.222.0.27 () {54 vars in 942 bytes} [Fri Jun 10 12:55:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:55:54.323566 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36e5ca3-eff7-4ed2-92e2-5d032669780a None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:55:54.323874 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36e5ca3-eff7-4ed2-92e2-5d032669780a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:55:54.324195 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36e5ca3-eff7-4ed2-92e2-5d032669780a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:55:54.377077 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-b36e5ca3-eff7-4ed2-92e2-5d032669780a None None] MFA Rules not processed for user `fb63b9cd74bf438aa25b72cef8337fec`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:55:54.382255 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b36e5ca3-eff7-4ed2-92e2-5d032669780a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:55:54.452236 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 706/1405] 10.222.0.27 () {58 vars in 1039 bytes} [Fri Jun 10 12:55:54 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 12:55:54.461861 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f0bc88-5f92-4387-b28f-dca82885daae None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:55:54.470108 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:55:54.476821 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f0bc88-5f92-4387-b28f-dca82885daae None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:55:54.482015 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d150ee8e-f7a0-4ab0-b4e7-d03b3174be7e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb63b9cd74bf438aa25b72cef8337fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:55:54.485253 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d150ee8e-f7a0-4ab0-b4e7-d03b3174be7e service swift] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:55:54.485253 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d150ee8e-f7a0-4ab0-b4e7-d03b3174be7e service swift] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:55:54.485253 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d150ee8e-f7a0-4ab0-b4e7-d03b3174be7e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:55:54.492415 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d150ee8e-f7a0-4ab0-b4e7-d03b3174be7e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:55:54.494002 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d150ee8e-f7a0-4ab0-b4e7-d03b3174be7e service swift] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:55:54.503021 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 700/1406] 10.222.0.27 () {60 vars in 1384 bytes} [Fri Jun 10 12:55:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 10 12:55:56.314244 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bdc99e-aa82-4be2-b1bc-1ba71c653976 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:55:56.314800 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bdc99e-aa82-4be2-b1bc-1ba71c653976 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:55:56.314800 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bdc99e-aa82-4be2-b1bc-1ba71c653976 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:55:56.316295 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 707/1407] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:55:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:55:56.321452 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905c70ef-3f1d-4565-adef-800d898b8243 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:55:56.321900 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905c70ef-3f1d-4565-adef-800d898b8243 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:55:56.322079 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905c70ef-3f1d-4565-adef-800d898b8243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:55:56.343768 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-905c70ef-3f1d-4565-adef-800d898b8243 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:55:56.345292 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-905c70ef-3f1d-4565-adef-800d898b8243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:55:56.354454 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 701/1408] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:55:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 12:55:56.398591 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bdc99e-aa82-4be2-b1bc-1ba71c653976 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:55:56.416071 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bdc99e-aa82-4be2-b1bc-1ba71c653976 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:55:56.422471 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae395e1-6a54-48b1-8189-1ad7ccb675e8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bd6caa461e84100a3d5fe5af5d80f3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:55:56.423237 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae395e1-6a54-48b1-8189-1ad7ccb675e8 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:55:56.423626 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae395e1-6a54-48b1-8189-1ad7ccb675e8 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:55:56.424075 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae395e1-6a54-48b1-8189-1ad7ccb675e8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:55:56.433063 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae395e1-6a54-48b1-8189-1ad7ccb675e8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:55:56.437789 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae395e1-6a54-48b1-8189-1ad7ccb675e8 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:55:56.445529 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 708/1409] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 12:55:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 12:56:06.337722 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=8 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:56:06.440758 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=62 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:56:18.040927 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9127084-44c8-4e45-b294-fb014c9f723e None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:56:18.041522 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9127084-44c8-4e45-b294-fb014c9f723e None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:56:18.041522 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9127084-44c8-4e45-b294-fb014c9f723e None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:56:18.042910 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 702/1410] 10.222.0.27 () {54 vars in 918 bytes} [Fri Jun 10 12:56:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:56:18.048424 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af94997-96f5-405a-b56b-a01815fb0e62 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:56:18.048827 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af94997-96f5-405a-b56b-a01815fb0e62 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:56:18.048977 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af94997-96f5-405a-b56b-a01815fb0e62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:56:18.066376 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-5af94997-96f5-405a-b56b-a01815fb0e62 None None] MFA Rules not processed for user `d972ab782b3f4d2aba575314f16dafc7`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:56:18.067573 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5af94997-96f5-405a-b56b-a01815fb0e62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:56:18.076426 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 709/1411] 10.222.0.27 () {58 vars in 1015 bytes} [Fri Jun 10 12:56:18 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 12:56:18.090578 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b388507-2990-41ce-acb5-b18b330fb8c1 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:56:18.090898 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b388507-2990-41ce-acb5-b18b330fb8c1 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:56:18.091177 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b388507-2990-41ce-acb5-b18b330fb8c1 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:56:18.093166 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 703/1412] 10.222.0.27 () {54 vars in 918 bytes} [Fri Jun 10 12:56:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:56:18.097706 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9eb401-f364-4520-92f8-9243e7c8af85 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:56:18.098000 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9eb401-f364-4520-92f8-9243e7c8af85 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:56:18.098281 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9eb401-f364-4520-92f8-9243e7c8af85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:56:18.116499 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-db9eb401-f364-4520-92f8-9243e7c8af85 None None] MFA Rules not processed for user `8bd6caa461e84100a3d5fe5af5d80f3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:56:18.117761 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-db9eb401-f364-4520-92f8-9243e7c8af85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:56:18.126876 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 710/1413] 10.222.0.27 () {58 vars in 1015 bytes} [Fri Jun 10 12:56:18 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 10 12:56:20.697374 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b62f4fe-f74d-4325-b0e8-f9b2679c8ec1 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:56:20.697916 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b62f4fe-f74d-4325-b0e8-f9b2679c8ec1 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:56:20.697916 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b62f4fe-f74d-4325-b0e8-f9b2679c8ec1 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:56:20.699234 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 704/1414] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 10 12:56:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:56:20.705099 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fafae27-d8e7-4267-9c0b-35165f7ef4a6 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:56:20.705501 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fafae27-d8e7-4267-9c0b-35165f7ef4a6 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:56:20.705662 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fafae27-d8e7-4267-9c0b-35165f7ef4a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:56:20.722795 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-5fafae27-d8e7-4267-9c0b-35165f7ef4a6 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:56:20.724001 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5fafae27-d8e7-4267-9c0b-35165f7ef4a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:56:20.732113 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 711/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 10 12:56:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 12:56:20.774180 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b62f4fe-f74d-4325-b0e8-f9b2679c8ec1 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:56:20.788274 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b62f4fe-f74d-4325-b0e8-f9b2679c8ec1 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:56:20.793497 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8056188-89bd-4b0d-9c90-97b5cb0a5afe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bd6caa461e84100a3d5fe5af5d80f3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:56:20.794124 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8056188-89bd-4b0d-9c90-97b5cb0a5afe service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:56:20.794413 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8056188-89bd-4b0d-9c90-97b5cb0a5afe service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:56:20.794731 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8056188-89bd-4b0d-9c90-97b5cb0a5afe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:56:20.802730 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8056188-89bd-4b0d-9c90-97b5cb0a5afe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:56:20.804333 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8056188-89bd-4b0d-9c90-97b5cb0a5afe service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:56:20.814004 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 705/1416] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 12:56:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 12:56:21.753236 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beec38bf-89a0-4e0c-a81c-2fbd1d9c2628 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:56:21.753804 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beec38bf-89a0-4e0c-a81c-2fbd1d9c2628 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:56:21.753804 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beec38bf-89a0-4e0c-a81c-2fbd1d9c2628 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:56:21.755203 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 712/1417] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 10 12:56:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:56:21.760197 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a447b99d-f7e7-4e9b-afcc-770cf2d915bc None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:56:21.760602 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a447b99d-f7e7-4e9b-afcc-770cf2d915bc None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:56:21.760725 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a447b99d-f7e7-4e9b-afcc-770cf2d915bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:56:21.778852 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-a447b99d-f7e7-4e9b-afcc-770cf2d915bc None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:56:21.780094 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a447b99d-f7e7-4e9b-afcc-770cf2d915bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:56:21.787942 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 706/1418] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 10 12:56:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 12:56:21.938276 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beec38bf-89a0-4e0c-a81c-2fbd1d9c2628 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:56:21.952212 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beec38bf-89a0-4e0c-a81c-2fbd1d9c2628 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:56:21.957469 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c51d81-1d7a-4211-8a52-2fe9c44b02e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bd6caa461e84100a3d5fe5af5d80f3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:56:21.958095 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c51d81-1d7a-4211-8a52-2fe9c44b02e6 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:56:21.958377 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c51d81-1d7a-4211-8a52-2fe9c44b02e6 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:56:21.958652 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c51d81-1d7a-4211-8a52-2fe9c44b02e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:56:21.966520 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c51d81-1d7a-4211-8a52-2fe9c44b02e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:56:21.968148 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c51d81-1d7a-4211-8a52-2fe9c44b02e6 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:56:21.977522 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 713/1419] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 12:56:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 12:56:22.938397 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9263938-ff4d-4d27-86fa-afdc8c915f71 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:56:22.938940 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9263938-ff4d-4d27-86fa-afdc8c915f71 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:56:22.938940 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9263938-ff4d-4d27-86fa-afdc8c915f71 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:56:22.940369 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 707/1420] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 10 12:56:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:56:22.945119 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e855482f-2feb-4a6a-ac89-4e059d42ff41 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:56:22.945573 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e855482f-2feb-4a6a-ac89-4e059d42ff41 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:56:22.945681 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e855482f-2feb-4a6a-ac89-4e059d42ff41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:56:22.963705 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-e855482f-2feb-4a6a-ac89-4e059d42ff41 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:56:22.964917 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e855482f-2feb-4a6a-ac89-4e059d42ff41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:56:22.972589 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 714/1421] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 10 12:56:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 12:56:23.123616 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9263938-ff4d-4d27-86fa-afdc8c915f71 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:56:23.137852 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9263938-ff4d-4d27-86fa-afdc8c915f71 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:56:23.143721 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cdd39b-92ab-4d38-a377-8fbc0cf316d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bd6caa461e84100a3d5fe5af5d80f3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:56:23.144380 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cdd39b-92ab-4d38-a377-8fbc0cf316d0 service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:56:23.144668 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cdd39b-92ab-4d38-a377-8fbc0cf316d0 service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:56:23.144948 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cdd39b-92ab-4d38-a377-8fbc0cf316d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:56:23.153011 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cdd39b-92ab-4d38-a377-8fbc0cf316d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:56:23.154626 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cdd39b-92ab-4d38-a377-8fbc0cf316d0 service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:56:23.164357 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 708/1422] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 12:56:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 12:56:24.061704 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327b42e0-412f-40d2-968f-396f648be4d8 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:56:24.062236 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327b42e0-412f-40d2-968f-396f648be4d8 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:56:24.062236 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327b42e0-412f-40d2-968f-396f648be4d8 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:56:24.063667 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 715/1423] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 10 12:56:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:56:24.068702 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd7cb09-9775-4721-989f-c8190cf88fa2 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:56:24.069106 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd7cb09-9775-4721-989f-c8190cf88fa2 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:56:24.069256 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd7cb09-9775-4721-989f-c8190cf88fa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:56:24.086694 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-bcd7cb09-9775-4721-989f-c8190cf88fa2 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:56:24.087893 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-bcd7cb09-9775-4721-989f-c8190cf88fa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:56:24.095395 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 709/1424] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 10 12:56:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 12:56:24.244316 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327b42e0-412f-40d2-968f-396f648be4d8 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:56:24.259054 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327b42e0-412f-40d2-968f-396f648be4d8 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:56:24.266813 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f31585f-ec42-4792-992a-40be464f5c6e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bd6caa461e84100a3d5fe5af5d80f3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:56:24.266813 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f31585f-ec42-4792-992a-40be464f5c6e service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:56:24.266813 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f31585f-ec42-4792-992a-40be464f5c6e service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:56:24.266813 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f31585f-ec42-4792-992a-40be464f5c6e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:56:24.273656 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f31585f-ec42-4792-992a-40be464f5c6e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:56:24.275218 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f31585f-ec42-4792-992a-40be464f5c6e service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:56:24.284779 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 716/1425] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 12:56:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 12:56:25.178380 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00de933-fc99-497b-90c3-958244e39142 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:56:25.178910 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00de933-fc99-497b-90c3-958244e39142 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:56:25.178910 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00de933-fc99-497b-90c3-958244e39142 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:56:25.180279 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 710/1426] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 10 12:56:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:56:25.187286 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cd5e3f-e0aa-4b67-acc8-5a4cacc56bee None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:56:25.187720 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cd5e3f-e0aa-4b67-acc8-5a4cacc56bee None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:56:25.187882 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cd5e3f-e0aa-4b67-acc8-5a4cacc56bee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:56:25.206110 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-96cd5e3f-e0aa-4b67-acc8-5a4cacc56bee None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:56:25.207317 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-96cd5e3f-e0aa-4b67-acc8-5a4cacc56bee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:56:25.214967 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 717/1427] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 10 12:56:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 12:56:25.364766 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00de933-fc99-497b-90c3-958244e39142 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:56:25.379650 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00de933-fc99-497b-90c3-958244e39142 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:56:25.385193 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4db8bb7-4ee5-432d-a9a8-7d3d5e7ec585 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bd6caa461e84100a3d5fe5af5d80f3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:56:25.385826 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4db8bb7-4ee5-432d-a9a8-7d3d5e7ec585 service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:56:25.386109 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4db8bb7-4ee5-432d-a9a8-7d3d5e7ec585 service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:56:25.386385 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4db8bb7-4ee5-432d-a9a8-7d3d5e7ec585 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:56:25.395084 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4db8bb7-4ee5-432d-a9a8-7d3d5e7ec585 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:56:25.396683 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4db8bb7-4ee5-432d-a9a8-7d3d5e7ec585 service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:56:25.405864 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 711/1428] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 12:56:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 12:56:26.335137 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc531ce8-f1d2-40f2-933c-21aefc30f52d None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:56:26.335666 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc531ce8-f1d2-40f2-933c-21aefc30f52d None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:56:26.335666 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc531ce8-f1d2-40f2-933c-21aefc30f52d None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:56:26.337058 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 718/1429] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 10 12:56:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:56:26.342441 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49081ed1-18b4-4598-982f-1f02d431a420 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:56:26.342890 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49081ed1-18b4-4598-982f-1f02d431a420 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:56:26.342996 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49081ed1-18b4-4598-982f-1f02d431a420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:56:26.360643 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-49081ed1-18b4-4598-982f-1f02d431a420 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:56:26.361829 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-49081ed1-18b4-4598-982f-1f02d431a420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:56:26.369604 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 712/1430] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 10 12:56:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 12:56:26.519785 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc531ce8-f1d2-40f2-933c-21aefc30f52d None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:56:26.534494 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc531ce8-f1d2-40f2-933c-21aefc30f52d None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:56:26.540038 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f8c41f-42f5-4918-a889-e7e7d43ece22 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bd6caa461e84100a3d5fe5af5d80f3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:56:26.540667 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f8c41f-42f5-4918-a889-e7e7d43ece22 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:56:26.540954 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f8c41f-42f5-4918-a889-e7e7d43ece22 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:56:26.541232 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f8c41f-42f5-4918-a889-e7e7d43ece22 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:56:26.549189 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f8c41f-42f5-4918-a889-e7e7d43ece22 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:56:26.550744 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f8c41f-42f5-4918-a889-e7e7d43ece22 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:56:26.560309 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 719/1431] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 12:56:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 12:56:27.479261 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16157435-2445-43d2-84c8-312e6507ffed None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:56:27.479801 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16157435-2445-43d2-84c8-312e6507ffed None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:56:27.479801 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16157435-2445-43d2-84c8-312e6507ffed None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:56:27.481214 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 713/1432] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 10 12:56:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:56:27.486648 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa06a97-4970-4d1f-bbe8-5db49a5804a3 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:56:27.487139 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa06a97-4970-4d1f-bbe8-5db49a5804a3 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:56:27.487251 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa06a97-4970-4d1f-bbe8-5db49a5804a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:56:27.504977 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-8fa06a97-4970-4d1f-bbe8-5db49a5804a3 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:56:27.506198 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8fa06a97-4970-4d1f-bbe8-5db49a5804a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:56:27.514105 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 720/1433] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 10 12:56:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 12:56:27.665230 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16157435-2445-43d2-84c8-312e6507ffed None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:56:27.679816 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16157435-2445-43d2-84c8-312e6507ffed None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:56:27.685083 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f18a807-2fae-4e91-bed2-91f11161ea2c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bd6caa461e84100a3d5fe5af5d80f3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:56:27.685711 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f18a807-2fae-4e91-bed2-91f11161ea2c service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:56:27.685994 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f18a807-2fae-4e91-bed2-91f11161ea2c service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:56:27.686269 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f18a807-2fae-4e91-bed2-91f11161ea2c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:56:27.694164 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f18a807-2fae-4e91-bed2-91f11161ea2c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:56:27.695726 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f18a807-2fae-4e91-bed2-91f11161ea2c service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:56:27.705210 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 714/1434] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 12:56:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 12:56:28.615953 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8502092-0ed3-46fc-ab3f-5a605b5a65c0 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:56:28.616502 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8502092-0ed3-46fc-ab3f-5a605b5a65c0 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:56:28.616619 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8502092-0ed3-46fc-ab3f-5a605b5a65c0 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:56:28.617897 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 721/1435] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 10 12:56:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:56:28.623528 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a0b263-f1a3-43b5-840f-7dfcd5013d39 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:56:28.623963 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a0b263-f1a3-43b5-840f-7dfcd5013d39 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:56:28.624122 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a0b263-f1a3-43b5-840f-7dfcd5013d39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:56:28.641519 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-93a0b263-f1a3-43b5-840f-7dfcd5013d39 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:56:28.642625 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-93a0b263-f1a3-43b5-840f-7dfcd5013d39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:56:28.650530 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 715/1436] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 10 12:56:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 12:56:28.799547 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8502092-0ed3-46fc-ab3f-5a605b5a65c0 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:56:28.813945 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8502092-0ed3-46fc-ab3f-5a605b5a65c0 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:56:28.819215 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c5afd6-81e8-4fdf-bea3-dd51737a18b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bd6caa461e84100a3d5fe5af5d80f3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:56:28.819838 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c5afd6-81e8-4fdf-bea3-dd51737a18b9 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:56:28.820146 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c5afd6-81e8-4fdf-bea3-dd51737a18b9 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:56:28.820425 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c5afd6-81e8-4fdf-bea3-dd51737a18b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:56:28.828348 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c5afd6-81e8-4fdf-bea3-dd51737a18b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:56:28.829898 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c5afd6-81e8-4fdf-bea3-dd51737a18b9 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:56:28.839002 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 722/1437] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 12:56:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 12:56:29.747684 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f29a4f-1673-49c1-b469-b7abb7c7474e None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:56:29.748276 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f29a4f-1673-49c1-b469-b7abb7c7474e None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:56:29.748276 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f29a4f-1673-49c1-b469-b7abb7c7474e None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:56:29.749567 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 716/1438] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 10 12:56:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:56:29.754460 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61183212-d25b-4dc4-a0f5-86c17fefdbfc None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:56:29.754881 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61183212-d25b-4dc4-a0f5-86c17fefdbfc None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:56:29.755017 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61183212-d25b-4dc4-a0f5-86c17fefdbfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:56:29.773082 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-61183212-d25b-4dc4-a0f5-86c17fefdbfc None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:56:29.774250 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-61183212-d25b-4dc4-a0f5-86c17fefdbfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:56:29.781809 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 723/1439] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 10 12:56:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 12:56:29.931986 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f29a4f-1673-49c1-b469-b7abb7c7474e None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:56:29.946909 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f29a4f-1673-49c1-b469-b7abb7c7474e None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:56:29.953009 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978e01a2-580e-4bda-9b05-ab6f1a73c381 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bd6caa461e84100a3d5fe5af5d80f3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:56:29.953644 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978e01a2-580e-4bda-9b05-ab6f1a73c381 service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:56:29.953931 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978e01a2-580e-4bda-9b05-ab6f1a73c381 service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:56:29.954211 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978e01a2-580e-4bda-9b05-ab6f1a73c381 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:56:29.962319 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978e01a2-580e-4bda-9b05-ab6f1a73c381 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:56:29.963930 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978e01a2-580e-4bda-9b05-ab6f1a73c381 service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:56:29.973449 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 717/1440] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 12:56:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 12:56:31.989232 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de3336f-ff4f-42fe-9b2d-aac30c45d453 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:56:31.989754 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de3336f-ff4f-42fe-9b2d-aac30c45d453 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:56:31.989869 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de3336f-ff4f-42fe-9b2d-aac30c45d453 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:56:31.991159 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 724/1441] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 10 12:56:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:56:31.996443 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef963e0-df8a-45bd-b4d5-9d1732db6a64 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:56:31.996875 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef963e0-df8a-45bd-b4d5-9d1732db6a64 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:56:31.997016 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef963e0-df8a-45bd-b4d5-9d1732db6a64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:56:32.014472 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-4ef963e0-df8a-45bd-b4d5-9d1732db6a64 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:56:32.015653 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-4ef963e0-df8a-45bd-b4d5-9d1732db6a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:56:32.023386 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 718/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 10 12:56:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 12:56:32.061442 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de3336f-ff4f-42fe-9b2d-aac30c45d453 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:56:32.071920 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de3336f-ff4f-42fe-9b2d-aac30c45d453 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:56:32.075339 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac564f4-5fe5-4a64-9048-77ff5d4da857 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:56:32.076007 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac564f4-5fe5-4a64-9048-77ff5d4da857 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:56:32.076291 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac564f4-5fe5-4a64-9048-77ff5d4da857 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:56:32.076567 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac564f4-5fe5-4a64-9048-77ff5d4da857 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:56:32.085137 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac564f4-5fe5-4a64-9048-77ff5d4da857 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:56:32.086562 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac564f4-5fe5-4a64-9048-77ff5d4da857 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:56:32.096202 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 725/1443] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 12:56:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 12:56:34.349784 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8352a948-4fab-4817-8c2c-37b9a903964b None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:56:34.350289 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8352a948-4fab-4817-8c2c-37b9a903964b None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:56:34.350383 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8352a948-4fab-4817-8c2c-37b9a903964b None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:56:34.351724 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 719/1444] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 10 12:56:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:56:34.357839 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f09d1e6-eba5-46b9-b4dc-384232941d4b None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:56:34.358272 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f09d1e6-eba5-46b9-b4dc-384232941d4b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:56:34.364634 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f09d1e6-eba5-46b9-b4dc-384232941d4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:56:34.375881 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-6f09d1e6-eba5-46b9-b4dc-384232941d4b None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:56:34.377206 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6f09d1e6-eba5-46b9-b4dc-384232941d4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:56:34.384929 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 726/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 10 12:56:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 12:56:34.423256 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8352a948-4fab-4817-8c2c-37b9a903964b None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:56:34.434077 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8352a948-4fab-4817-8c2c-37b9a903964b None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:56:34.437588 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97853647-9843-4708-bb54-a36266810cf9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:56:34.438250 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97853647-9843-4708-bb54-a36266810cf9 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:56:34.438535 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97853647-9843-4708-bb54-a36266810cf9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:56:34.438809 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97853647-9843-4708-bb54-a36266810cf9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:56:34.447564 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97853647-9843-4708-bb54-a36266810cf9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:56:34.450625 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97853647-9843-4708-bb54-a36266810cf9 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:56:34.458415 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 720/1446] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 12:56:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 12:56:36.947249 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bd4724-e6c6-4286-b3bd-27f66d5f8b29 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:56:36.947684 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bd4724-e6c6-4286-b3bd-27f66d5f8b29 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:56:36.947791 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bd4724-e6c6-4286-b3bd-27f66d5f8b29 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:56:36.949200 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 727/1447] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 10 12:56:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:56:36.954897 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce530911-224e-4ebb-8d67-3e3450252aae None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:56:36.955296 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce530911-224e-4ebb-8d67-3e3450252aae None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:56:36.955456 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce530911-224e-4ebb-8d67-3e3450252aae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:56:36.973463 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-ce530911-224e-4ebb-8d67-3e3450252aae None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:56:36.974648 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-ce530911-224e-4ebb-8d67-3e3450252aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:56:36.982291 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 721/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 10 12:56:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 12:56:37.019505 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bd4724-e6c6-4286-b3bd-27f66d5f8b29 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:56:37.030574 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bd4724-e6c6-4286-b3bd-27f66d5f8b29 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:56:37.034059 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19614004-654d-4ef0-93b2-f58030bcea1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:56:37.034686 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19614004-654d-4ef0-93b2-f58030bcea1a service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:56:37.035230 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19614004-654d-4ef0-93b2-f58030bcea1a service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:56:37.035509 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19614004-654d-4ef0-93b2-f58030bcea1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:56:37.044181 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19614004-654d-4ef0-93b2-f58030bcea1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:56:37.045596 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19614004-654d-4ef0-93b2-f58030bcea1a service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:56:37.054834 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 728/1449] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 12:56:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 12:56:39.281021 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f805051f-dd8d-4669-8871-ee23b553a20e None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:56:39.281585 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f805051f-dd8d-4669-8871-ee23b553a20e None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:56:39.281585 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f805051f-dd8d-4669-8871-ee23b553a20e None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:56:39.282878 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 722/1450] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 10 12:56:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:56:39.288536 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139a3ad0-bfb1-4ab4-a40a-2672b3af2e20 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:56:39.288977 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139a3ad0-bfb1-4ab4-a40a-2672b3af2e20 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:56:39.289191 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139a3ad0-bfb1-4ab4-a40a-2672b3af2e20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:56:39.307805 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-139a3ad0-bfb1-4ab4-a40a-2672b3af2e20 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:56:39.309133 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-139a3ad0-bfb1-4ab4-a40a-2672b3af2e20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:56:39.317126 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 729/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 10 12:56:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 12:56:39.355434 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f805051f-dd8d-4669-8871-ee23b553a20e None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:56:39.357117 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f805051f-dd8d-4669-8871-ee23b553a20e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:56:39.369608 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f805051f-dd8d-4669-8871-ee23b553a20e None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:56:39.373012 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac560f6-1591-49f1-9885-a51f59435f94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:56:39.373636 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac560f6-1591-49f1-9885-a51f59435f94 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:56:39.373919 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac560f6-1591-49f1-9885-a51f59435f94 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:56:39.374203 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac560f6-1591-49f1-9885-a51f59435f94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:56:39.382722 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac560f6-1591-49f1-9885-a51f59435f94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:56:39.384165 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac560f6-1591-49f1-9885-a51f59435f94 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:56:39.393839 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 723/1452] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 12:56:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 12:56:41.624719 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b570d96-f5a9-4e41-ad3e-7530b534da3e None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:56:41.625278 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b570d96-f5a9-4e41-ad3e-7530b534da3e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:56:41.625278 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b570d96-f5a9-4e41-ad3e-7530b534da3e None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:56:41.626628 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 730/1453] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 10 12:56:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:56:41.632052 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57138284-c661-4e33-84a5-00373c4b2be3 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:56:41.632475 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57138284-c661-4e33-84a5-00373c4b2be3 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:56:41.632605 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57138284-c661-4e33-84a5-00373c4b2be3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:56:41.650008 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-57138284-c661-4e33-84a5-00373c4b2be3 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:56:41.651198 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-57138284-c661-4e33-84a5-00373c4b2be3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:56:41.659005 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 724/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 10 12:56:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 12:56:41.697075 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b570d96-f5a9-4e41-ad3e-7530b534da3e None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:56:41.707795 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b570d96-f5a9-4e41-ad3e-7530b534da3e None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:56:41.711282 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfba89c0-250d-407e-a29c-5cbe063d00ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:56:41.711941 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfba89c0-250d-407e-a29c-5cbe063d00ba service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:56:41.712227 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfba89c0-250d-407e-a29c-5cbe063d00ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:56:41.712505 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfba89c0-250d-407e-a29c-5cbe063d00ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:56:41.721698 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfba89c0-250d-407e-a29c-5cbe063d00ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:56:41.723110 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfba89c0-250d-407e-a29c-5cbe063d00ba service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:56:41.732929 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 731/1455] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 12:56:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 12:56:43.899405 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4150dc-70bb-4d72-818a-5065d14d4658 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:56:43.899916 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4150dc-70bb-4d72-818a-5065d14d4658 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:56:43.900046 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4150dc-70bb-4d72-818a-5065d14d4658 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:56:43.901386 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 725/1456] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 10 12:56:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:56:43.906614 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41187fa-3c53-4c0b-b2ba-6b87fe269cd3 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:56:43.907032 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41187fa-3c53-4c0b-b2ba-6b87fe269cd3 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:56:43.907175 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41187fa-3c53-4c0b-b2ba-6b87fe269cd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:56:43.925452 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-f41187fa-3c53-4c0b-b2ba-6b87fe269cd3 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:56:43.926645 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f41187fa-3c53-4c0b-b2ba-6b87fe269cd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:56:43.935199 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 732/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 10 12:56:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 12:56:43.972459 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4150dc-70bb-4d72-818a-5065d14d4658 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:56:43.983587 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4150dc-70bb-4d72-818a-5065d14d4658 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:56:43.987022 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293403c4-d97b-4193-bc61-56910d32e173 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:56:43.987643 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293403c4-d97b-4193-bc61-56910d32e173 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:56:43.987955 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293403c4-d97b-4193-bc61-56910d32e173 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:56:43.988232 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293403c4-d97b-4193-bc61-56910d32e173 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:56:43.996893 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293403c4-d97b-4193-bc61-56910d32e173 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:56:43.998316 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293403c4-d97b-4193-bc61-56910d32e173 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:56:44.007447 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 726/1458] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 12:56:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 12:56:46.135700 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395975e7-618f-4429-8255-ced6f94f818d None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:56:46.136253 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395975e7-618f-4429-8255-ced6f94f818d None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:56:46.136253 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395975e7-618f-4429-8255-ced6f94f818d None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:56:46.138151 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 733/1459] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 10 12:56:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:56:46.143490 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5fa6a6-d833-48f3-9ae4-75cd2ca46743 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:56:46.143955 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5fa6a6-d833-48f3-9ae4-75cd2ca46743 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:56:46.144064 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5fa6a6-d833-48f3-9ae4-75cd2ca46743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:56:46.162132 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-3b5fa6a6-d833-48f3-9ae4-75cd2ca46743 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:56:46.163327 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-3b5fa6a6-d833-48f3-9ae4-75cd2ca46743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:56:46.170947 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 727/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 10 12:56:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 12:56:46.209418 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395975e7-618f-4429-8255-ced6f94f818d None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:56:46.220057 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395975e7-618f-4429-8255-ced6f94f818d None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:56:46.223507 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cd5cbd-39ec-49f7-b6c4-986edc30a8e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:56:46.224178 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cd5cbd-39ec-49f7-b6c4-986edc30a8e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:56:46.224462 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cd5cbd-39ec-49f7-b6c4-986edc30a8e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:56:46.224741 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cd5cbd-39ec-49f7-b6c4-986edc30a8e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:56:46.233478 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cd5cbd-39ec-49f7-b6c4-986edc30a8e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:56:46.234880 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cd5cbd-39ec-49f7-b6c4-986edc30a8e9 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:56:46.243997 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 734/1461] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 12:56:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 12:56:48.412774 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45481321-ac9d-43c4-9b49-aba92a6513e9 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:56:48.413202 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45481321-ac9d-43c4-9b49-aba92a6513e9 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:56:48.413391 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45481321-ac9d-43c4-9b49-aba92a6513e9 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:56:48.414795 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 728/1462] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 10 12:56:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:56:48.420446 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06526872-1a10-4a18-946c-130b8d1b46d0 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:56:48.420862 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06526872-1a10-4a18-946c-130b8d1b46d0 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:56:48.420972 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06526872-1a10-4a18-946c-130b8d1b46d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:56:48.426872 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=79 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:56:48.440166 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-06526872-1a10-4a18-946c-130b8d1b46d0 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:56:48.441373 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-06526872-1a10-4a18-946c-130b8d1b46d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:56:48.449177 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 735/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 10 12:56:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 12:56:48.487584 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45481321-ac9d-43c4-9b49-aba92a6513e9 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:56:48.493244 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=73 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:56:48.499135 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45481321-ac9d-43c4-9b49-aba92a6513e9 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:56:48.502654 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c944b0-102c-4445-80ab-68e06c70660d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:56:48.503333 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c944b0-102c-4445-80ab-68e06c70660d service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:56:48.503622 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c944b0-102c-4445-80ab-68e06c70660d service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:56:48.503945 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c944b0-102c-4445-80ab-68e06c70660d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:56:48.512882 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c944b0-102c-4445-80ab-68e06c70660d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:56:48.514303 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c944b0-102c-4445-80ab-68e06c70660d service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:56:48.524238 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 729/1464] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 12:56:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 12:56:50.645585 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e673b59-0318-4db5-8896-b7e2ae4499ae None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:56:50.646139 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e673b59-0318-4db5-8896-b7e2ae4499ae None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:56:50.646139 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e673b59-0318-4db5-8896-b7e2ae4499ae None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:56:50.647528 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 736/1465] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 10 12:56:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:56:50.653367 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3d27e7-acdc-4fa4-a919-64f35f35342e None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:56:50.653792 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3d27e7-acdc-4fa4-a919-64f35f35342e None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:56:50.653888 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3d27e7-acdc-4fa4-a919-64f35f35342e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:56:50.671336 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-2f3d27e7-acdc-4fa4-a919-64f35f35342e None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:56:50.672543 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-2f3d27e7-acdc-4fa4-a919-64f35f35342e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:56:50.680074 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 730/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 10 12:56:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 12:56:50.716333 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e673b59-0318-4db5-8896-b7e2ae4499ae None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:56:50.726782 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e673b59-0318-4db5-8896-b7e2ae4499ae None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:56:50.730217 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b27659-6d89-4c43-9da0-bbb1acd71c5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:56:50.730867 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b27659-6d89-4c43-9da0-bbb1acd71c5f service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:56:50.731146 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b27659-6d89-4c43-9da0-bbb1acd71c5f service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:56:50.731418 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b27659-6d89-4c43-9da0-bbb1acd71c5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:56:50.740067 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b27659-6d89-4c43-9da0-bbb1acd71c5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:56:50.741465 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b27659-6d89-4c43-9da0-bbb1acd71c5f service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:56:50.751388 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 737/1467] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 12:56:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 12:56:52.853704 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f9d1e3-8ef0-47f9-8f76-975763f0f970 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:56:52.854240 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f9d1e3-8ef0-47f9-8f76-975763f0f970 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:56:52.854240 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f9d1e3-8ef0-47f9-8f76-975763f0f970 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:56:52.855562 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 731/1468] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 10 12:56:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:56:52.861047 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73ed2db-fcdc-465f-a436-a71447408407 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:56:52.861667 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73ed2db-fcdc-465f-a436-a71447408407 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:56:52.861667 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73ed2db-fcdc-465f-a436-a71447408407 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:56:52.879447 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-b73ed2db-fcdc-465f-a436-a71447408407 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:56:52.880763 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b73ed2db-fcdc-465f-a436-a71447408407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:56:52.888957 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 738/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 10 12:56:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 12:56:52.926275 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f9d1e3-8ef0-47f9-8f76-975763f0f970 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:56:52.937518 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f9d1e3-8ef0-47f9-8f76-975763f0f970 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:56:52.941007 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86a2af6-708e-4552-b1f9-8bc4b83af2f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:56:52.941638 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86a2af6-708e-4552-b1f9-8bc4b83af2f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:56:52.941926 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86a2af6-708e-4552-b1f9-8bc4b83af2f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:56:52.942208 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86a2af6-708e-4552-b1f9-8bc4b83af2f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:56:52.951733 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86a2af6-708e-4552-b1f9-8bc4b83af2f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:56:52.953205 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86a2af6-708e-4552-b1f9-8bc4b83af2f5 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:56:52.962630 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 732/1470] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 12:56:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 12:57:02.874197 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=11 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:57:02.958423 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=14 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:57:12.953663 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98cbb29-e29a-4197-858e-97a90610f2a3 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:57:12.954265 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98cbb29-e29a-4197-858e-97a90610f2a3 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:57:12.954265 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98cbb29-e29a-4197-858e-97a90610f2a3 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:57:12.955614 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 739/1471] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:57:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:57:12.960661 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d25643a-e093-4bd8-b4f3-4eac9020a8bd None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:57:12.961091 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d25643a-e093-4bd8-b4f3-4eac9020a8bd None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:57:12.961225 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d25643a-e093-4bd8-b4f3-4eac9020a8bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:57:12.979281 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-5d25643a-e093-4bd8-b4f3-4eac9020a8bd None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:57:12.980501 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5d25643a-e093-4bd8-b4f3-4eac9020a8bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:57:12.988147 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 733/1472] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:57:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 12:57:13.035282 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987ca243-b163-4298-b35a-f17b545a89a3 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:57:13.035578 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987ca243-b163-4298-b35a-f17b545a89a3 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:57:13.035877 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987ca243-b163-4298-b35a-f17b545a89a3 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:57:13.037257 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 740/1473] 10.222.0.27 () {54 vars in 962 bytes} [Fri Jun 10 12:57:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:57:13.042125 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b20273b-efb8-41a7-8c4c-cbb07e2f6a27 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:57:13.042447 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b20273b-efb8-41a7-8c4c-cbb07e2f6a27 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:57:13.042727 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b20273b-efb8-41a7-8c4c-cbb07e2f6a27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:57:13.060934 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-4b20273b-efb8-41a7-8c4c-cbb07e2f6a27 None None] MFA Rules not processed for user `6d3eb01a98274050b930616b98998c06`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:57:13.062108 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-4b20273b-efb8-41a7-8c4c-cbb07e2f6a27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:57:13.068681 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 734/1474] 10.222.0.27 () {58 vars in 1059 bytes} [Fri Jun 10 12:57:13 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 12:57:13.076516 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987ca243-b163-4298-b35a-f17b545a89a3 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:57:13.089149 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987ca243-b163-4298-b35a-f17b545a89a3 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:57:13.093269 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836470e9-01bf-4edd-af30-ebedd48df596 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d3eb01a98274050b930616b98998c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:57:13.093905 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836470e9-01bf-4edd-af30-ebedd48df596 service glance] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:57:13.094190 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836470e9-01bf-4edd-af30-ebedd48df596 service glance] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:57:13.094468 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836470e9-01bf-4edd-af30-ebedd48df596 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:57:13.103330 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836470e9-01bf-4edd-af30-ebedd48df596 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:57:13.104777 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836470e9-01bf-4edd-af30-ebedd48df596 service glance] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:57:13.114069 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 741/1475] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 12:57:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 12:57:15.280744 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd493fb-445e-470c-a630-dadc8d5d0140 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:57:15.281306 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd493fb-445e-470c-a630-dadc8d5d0140 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:57:15.281306 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd493fb-445e-470c-a630-dadc8d5d0140 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:57:15.282609 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 735/1476] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:57:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:57:15.287786 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93ea77f-ea19-45b6-a162-21d084e67223 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:57:15.288257 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93ea77f-ea19-45b6-a162-21d084e67223 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:57:15.288354 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93ea77f-ea19-45b6-a162-21d084e67223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:57:15.306170 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-c93ea77f-ea19-45b6-a162-21d084e67223 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:57:15.307351 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c93ea77f-ea19-45b6-a162-21d084e67223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:57:15.315603 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 742/1477] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:57:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 12:57:15.338712 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd493fb-445e-470c-a630-dadc8d5d0140 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:57:15.350890 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd493fb-445e-470c-a630-dadc8d5d0140 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:57:15.355896 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b201c12-ade7-4fc2-91ef-dd97bb0d5b88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:57:15.356539 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b201c12-ade7-4fc2-91ef-dd97bb0d5b88 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:57:15.356822 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b201c12-ade7-4fc2-91ef-dd97bb0d5b88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:57:15.357097 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b201c12-ade7-4fc2-91ef-dd97bb0d5b88 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:57:15.357830 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b201c12-ade7-4fc2-91ef-dd97bb0d5b88 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:57:15.359588 dv-cinder-master-3-smb devstack@keystone.service[80238]: WARNING py.warnings [None req-4b201c12-ade7-4fc2-91ef-dd97bb0d5b88 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 10 12:57:15.359588 dv-cinder-master-3-smb devstack@keystone.service[80238]: warnings.warn(msg) Jun 10 12:57:15.359588 dv-cinder-master-3-smb devstack@keystone.service[80238]: Jun 10 12:57:15.360747 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b201c12-ade7-4fc2-91ef-dd97bb0d5b88 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:57:15.380103 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 736/1478] 10.222.0.27 () {56 vars in 1108 bytes} [Fri Jun 10 12:57:15 2022] GET /identity/v3/projects => generated 2553 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 12:57:17.490075 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57e54fb-3eb6-4dd5-86bd-4266bc599247 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:57:17.490644 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57e54fb-3eb6-4dd5-86bd-4266bc599247 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:57:17.490644 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57e54fb-3eb6-4dd5-86bd-4266bc599247 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:57:17.492019 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 743/1479] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:57:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:57:17.497247 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdc3b04-0ab8-4d35-a015-e8e82d964d97 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:57:17.497644 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdc3b04-0ab8-4d35-a015-e8e82d964d97 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:57:17.497810 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdc3b04-0ab8-4d35-a015-e8e82d964d97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:57:17.515737 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-cbdc3b04-0ab8-4d35-a015-e8e82d964d97 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:57:17.516987 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-cbdc3b04-0ab8-4d35-a015-e8e82d964d97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:57:17.524665 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 737/1480] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:57:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 12:57:17.567008 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57e54fb-3eb6-4dd5-86bd-4266bc599247 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:57:17.581448 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57e54fb-3eb6-4dd5-86bd-4266bc599247 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:57:17.592773 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ef51d0-b00f-477b-a363-5c0266a677ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bd6caa461e84100a3d5fe5af5d80f3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:57:17.592773 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ef51d0-b00f-477b-a363-5c0266a677ff service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:57:17.592773 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ef51d0-b00f-477b-a363-5c0266a677ff service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:57:17.592773 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ef51d0-b00f-477b-a363-5c0266a677ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:57:17.596278 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ef51d0-b00f-477b-a363-5c0266a677ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:57:17.597938 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ef51d0-b00f-477b-a363-5c0266a677ff service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:57:17.608518 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 744/1481] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 12:57:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 12:57:19.898108 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e514c-7fbd-4e74-b355-5b98197aefc1 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:57:19.898579 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e514c-7fbd-4e74-b355-5b98197aefc1 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:57:19.898678 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e514c-7fbd-4e74-b355-5b98197aefc1 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:57:19.900070 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 738/1482] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:57:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:57:19.905759 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b7d9ed-ff59-4977-83b7-86c8c42fdc8f None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:57:19.906191 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b7d9ed-ff59-4977-83b7-86c8c42fdc8f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:57:19.906314 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b7d9ed-ff59-4977-83b7-86c8c42fdc8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:57:19.923747 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-a7b7d9ed-ff59-4977-83b7-86c8c42fdc8f None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:57:19.924967 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a7b7d9ed-ff59-4977-83b7-86c8c42fdc8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:57:19.933133 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 745/1483] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:57:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 12:57:19.969886 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5e514c-7fbd-4e74-b355-5b98197aefc1 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:57:19.980494 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5e514c-7fbd-4e74-b355-5b98197aefc1 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:57:19.983963 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c13486f-c358-40da-9899-8d288e252971 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:57:19.984596 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c13486f-c358-40da-9899-8d288e252971 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:57:19.984882 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c13486f-c358-40da-9899-8d288e252971 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:57:19.985159 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c13486f-c358-40da-9899-8d288e252971 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:57:19.993886 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c13486f-c358-40da-9899-8d288e252971 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:57:19.995815 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c13486f-c358-40da-9899-8d288e252971 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:57:20.005141 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 739/1484] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 12:57:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 12:57:22.168035 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9a9d46-ac27-4e91-9bf9-cb84fc3942d9 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:57:22.168582 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9a9d46-ac27-4e91-9bf9-cb84fc3942d9 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:57:22.168582 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9a9d46-ac27-4e91-9bf9-cb84fc3942d9 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:57:22.169991 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 746/1485] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:57:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:57:22.175269 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1858e22e-1ec2-4224-8993-78c59132c921 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:57:22.175663 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1858e22e-1ec2-4224-8993-78c59132c921 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:57:22.175840 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1858e22e-1ec2-4224-8993-78c59132c921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:57:22.193189 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-1858e22e-1ec2-4224-8993-78c59132c921 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:57:22.194379 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-1858e22e-1ec2-4224-8993-78c59132c921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:57:22.202043 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 740/1486] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:57:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 12:57:22.240058 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9a9d46-ac27-4e91-9bf9-cb84fc3942d9 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:57:22.250506 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9a9d46-ac27-4e91-9bf9-cb84fc3942d9 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:57:22.253931 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc93c78d-1590-41f7-b341-21b8f282d938 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:57:22.254553 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc93c78d-1590-41f7-b341-21b8f282d938 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:57:22.254833 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc93c78d-1590-41f7-b341-21b8f282d938 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:57:22.255106 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc93c78d-1590-41f7-b341-21b8f282d938 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:57:22.263670 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc93c78d-1590-41f7-b341-21b8f282d938 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:57:22.265143 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc93c78d-1590-41f7-b341-21b8f282d938 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:57:22.274360 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 747/1487] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 12:57:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 12:57:24.469018 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5668e28-88a5-471c-b717-33aca2a9db15 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:57:24.469580 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5668e28-88a5-471c-b717-33aca2a9db15 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:57:24.469580 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5668e28-88a5-471c-b717-33aca2a9db15 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:57:24.470951 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 741/1488] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:57:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:57:24.476394 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024338f2-3036-427f-bdf0-4387e2b558cf None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:57:24.476802 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024338f2-3036-427f-bdf0-4387e2b558cf None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:57:24.476977 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024338f2-3036-427f-bdf0-4387e2b558cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:57:24.494300 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-024338f2-3036-427f-bdf0-4387e2b558cf None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:57:24.496038 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-024338f2-3036-427f-bdf0-4387e2b558cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:57:24.503705 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 748/1489] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:57:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 10 12:57:24.544585 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5668e28-88a5-471c-b717-33aca2a9db15 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:57:24.554889 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5668e28-88a5-471c-b717-33aca2a9db15 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:57:24.558837 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895ea697-582f-401a-9206-c7da1b9e626e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:57:24.559466 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895ea697-582f-401a-9206-c7da1b9e626e service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:57:24.559747 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895ea697-582f-401a-9206-c7da1b9e626e service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:57:24.560054 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895ea697-582f-401a-9206-c7da1b9e626e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:57:24.568735 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895ea697-582f-401a-9206-c7da1b9e626e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:57:24.570152 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895ea697-582f-401a-9206-c7da1b9e626e service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:57:24.580068 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 742/1490] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 12:57:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 12:57:27.177434 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b5e205-fc05-4dc0-bf57-979e91304162 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:57:27.177959 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b5e205-fc05-4dc0-bf57-979e91304162 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:57:27.177959 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b5e205-fc05-4dc0-bf57-979e91304162 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:57:27.179263 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 749/1491] 10.222.0.27 () {54 vars in 926 bytes} [Fri Jun 10 12:57:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:57:27.184575 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304ed83a-ebb6-4f56-950c-305f713c6faa None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:57:27.185002 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304ed83a-ebb6-4f56-950c-305f713c6faa None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:57:27.185118 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304ed83a-ebb6-4f56-950c-305f713c6faa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:57:27.202832 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-304ed83a-ebb6-4f56-950c-305f713c6faa None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:57:27.204045 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-304ed83a-ebb6-4f56-950c-305f713c6faa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:57:27.211588 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 743/1492] 10.222.0.27 () {58 vars in 1023 bytes} [Fri Jun 10 12:57:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 10 12:57:27.249395 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b5e205-fc05-4dc0-bf57-979e91304162 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:57:27.259734 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b5e205-fc05-4dc0-bf57-979e91304162 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:57:27.263735 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce14d73e-dbaf-42d3-b03a-2c773b63da7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:57:27.264395 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce14d73e-dbaf-42d3-b03a-2c773b63da7d service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:57:27.264675 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce14d73e-dbaf-42d3-b03a-2c773b63da7d service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:57:27.264948 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce14d73e-dbaf-42d3-b03a-2c773b63da7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:57:27.273451 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce14d73e-dbaf-42d3-b03a-2c773b63da7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:57:27.274856 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce14d73e-dbaf-42d3-b03a-2c773b63da7d service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:57:27.284758 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 750/1493] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 12:57:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 10 12:57:37.197464 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=33 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:57:37.280414 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=33 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:59:21.812735 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492332f7-306b-472d-9d3f-c770c22aece6 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:59:21.813353 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492332f7-306b-472d-9d3f-c770c22aece6 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:59:21.813353 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492332f7-306b-472d-9d3f-c770c22aece6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:59:21.830962 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-492332f7-306b-472d-9d3f-c770c22aece6 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:59:21.832181 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-492332f7-306b-472d-9d3f-c770c22aece6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:59:21.839814 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 744/1494] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 12:59:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 12:59:21.848849 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c45c3a-c3a3-47a9-ab11-332dffc6e90e None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:59:21.849359 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c45c3a-c3a3-47a9-ab11-332dffc6e90e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:59:21.849466 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c45c3a-c3a3-47a9-ab11-332dffc6e90e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:59:21.868231 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-15c45c3a-c3a3-47a9-ab11-332dffc6e90e None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:59:21.869427 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-15c45c3a-c3a3-47a9-ab11-332dffc6e90e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:59:21.877195 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 751/1495] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 12:59:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 12:59:21.881679 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492332f7-306b-472d-9d3f-c770c22aece6 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:59:21.891394 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492332f7-306b-472d-9d3f-c770c22aece6 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:59:21.895826 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e523eb31-1c3c-43ac-a412-e4414ef96449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:59:21.896509 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e523eb31-1c3c-43ac-a412-e4414ef96449 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:59:21.896796 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e523eb31-1c3c-43ac-a412-e4414ef96449 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:59:21.897072 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e523eb31-1c3c-43ac-a412-e4414ef96449 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:59:21.897821 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e523eb31-1c3c-43ac-a412-e4414ef96449 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 12:59:21.898153 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e523eb31-1c3c-43ac-a412-e4414ef96449 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:59:21.899170 dv-cinder-master-3-smb devstack@keystone.service[80238]: WARNING py.warnings [None req-e523eb31-1c3c-43ac-a412-e4414ef96449 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 10 12:59:21.899170 dv-cinder-master-3-smb devstack@keystone.service[80238]: warnings.warn(msg) Jun 10 12:59:21.899170 dv-cinder-master-3-smb devstack@keystone.service[80238]: Jun 10 12:59:21.900061 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e523eb31-1c3c-43ac-a412-e4414ef96449 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:59:21.911725 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 745/1496] 10.222.0.27 () {58 vars in 1152 bytes} [Fri Jun 10 12:59:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 12:59:21.920073 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c45c3a-c3a3-47a9-ab11-332dffc6e90e None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:59:21.929153 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c45c3a-c3a3-47a9-ab11-332dffc6e90e None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:59:21.934224 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1568eb-8f16-4bc9-8bd6-56b6fa17f04d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:59:21.934866 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1568eb-8f16-4bc9-8bd6-56b6fa17f04d admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:59:21.935152 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1568eb-8f16-4bc9-8bd6-56b6fa17f04d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:59:21.935431 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1568eb-8f16-4bc9-8bd6-56b6fa17f04d admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:59:21.936155 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1568eb-8f16-4bc9-8bd6-56b6fa17f04d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:59:21.937919 dv-cinder-master-3-smb devstack@keystone.service[80237]: WARNING py.warnings [None req-eb1568eb-8f16-4bc9-8bd6-56b6fa17f04d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 10 12:59:21.937919 dv-cinder-master-3-smb devstack@keystone.service[80237]: warnings.warn(msg) Jun 10 12:59:21.937919 dv-cinder-master-3-smb devstack@keystone.service[80237]: Jun 10 12:59:21.939095 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1568eb-8f16-4bc9-8bd6-56b6fa17f04d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:59:22.045997 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 752/1497] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 12:59:21 2022] POST /identity/v3/projects => generated 379 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 10 12:59:22.050284 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e523eb31-1c3c-43ac-a412-e4414ef96449 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:59:22.060722 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e523eb31-1c3c-43ac-a412-e4414ef96449 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:59:22.115258 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f9d623-59f7-4be4-89bf-6b396b6eb67d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:59:22.115956 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f9d623-59f7-4be4-89bf-6b396b6eb67d admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:59:22.116246 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f9d623-59f7-4be4-89bf-6b396b6eb67d admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:59:22.116532 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f9d623-59f7-4be4-89bf-6b396b6eb67d admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:59:22.117244 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f9d623-59f7-4be4-89bf-6b396b6eb67d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:59:22.118265 dv-cinder-master-3-smb devstack@keystone.service[80238]: WARNING py.warnings [None req-09f9d623-59f7-4be4-89bf-6b396b6eb67d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 10 12:59:22.118265 dv-cinder-master-3-smb devstack@keystone.service[80238]: warnings.warn(msg) Jun 10 12:59:22.118265 dv-cinder-master-3-smb devstack@keystone.service[80238]: Jun 10 12:59:22.119463 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f9d623-59f7-4be4-89bf-6b396b6eb67d admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:59:22.206345 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 746/1498] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 12:59:22 2022] POST /identity/v3/users => generated 290 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 12:59:22.210435 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1568eb-8f16-4bc9-8bd6-56b6fa17f04d admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:59:22.221194 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1568eb-8f16-4bc9-8bd6-56b6fa17f04d admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:59:22.225695 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ec8584-7bcc-4088-a74a-544f3137594b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:59:22.226942 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ec8584-7bcc-4088-a74a-544f3137594b admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:59:22.227226 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ec8584-7bcc-4088-a74a-544f3137594b admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:59:22.227597 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ec8584-7bcc-4088-a74a-544f3137594b admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:59:22.228356 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ec8584-7bcc-4088-a74a-544f3137594b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:59:22.229351 dv-cinder-master-3-smb devstack@keystone.service[80237]: WARNING py.warnings [None req-b2ec8584-7bcc-4088-a74a-544f3137594b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 10 12:59:22.229351 dv-cinder-master-3-smb devstack@keystone.service[80237]: warnings.warn(msg) Jun 10 12:59:22.229351 dv-cinder-master-3-smb devstack@keystone.service[80237]: Jun 10 12:59:22.230221 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ec8584-7bcc-4088-a74a-544f3137594b admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:59:22.242810 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 753/1499] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 12:59:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 12:59:22.246602 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f9d623-59f7-4be4-89bf-6b396b6eb67d admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:59:22.256272 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f9d623-59f7-4be4-89bf-6b396b6eb67d admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:59:22.260713 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a58ae8-6ba9-437b-bca9-a6ba93d7b4a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:59:22.261359 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a58ae8-6ba9-437b-bca9-a6ba93d7b4a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:59:22.261643 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a58ae8-6ba9-437b-bca9-a6ba93d7b4a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:59:22.261950 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a58ae8-6ba9-437b-bca9-a6ba93d7b4a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:59:22.262671 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a58ae8-6ba9-437b-bca9-a6ba93d7b4a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:59:22.263676 dv-cinder-master-3-smb devstack@keystone.service[80238]: WARNING py.warnings [None req-19a58ae8-6ba9-437b-bca9-a6ba93d7b4a1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 10 12:59:22.263676 dv-cinder-master-3-smb devstack@keystone.service[80238]: warnings.warn(msg) Jun 10 12:59:22.263676 dv-cinder-master-3-smb devstack@keystone.service[80238]: Jun 10 12:59:22.264586 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a58ae8-6ba9-437b-bca9-a6ba93d7b4a1 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:59:22.278065 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 747/1500] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 12:59:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 12:59:22.282205 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ec8584-7bcc-4088-a74a-544f3137594b admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:59:22.292360 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ec8584-7bcc-4088-a74a-544f3137594b admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:59:22.296796 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87466cec-5783-4adb-8fc5-6269ec6a9419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:59:22.297645 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87466cec-5783-4adb-8fc5-6269ec6a9419 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:59:22.297927 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87466cec-5783-4adb-8fc5-6269ec6a9419 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:59:22.298202 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87466cec-5783-4adb-8fc5-6269ec6a9419 admin admin] PATH_INFO: `/v3/projects/4b63dd2efe784b06a622cb89e6550327/users/63f8822cc1ff4e4ca6a14ce9c4f9a39a/roles/43106c664e1b40de984c0c62701149c8` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:59:22.312324 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87466cec-5783-4adb-8fc5-6269ec6a9419 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b63dd2efe784b06a622cb89e6550327, user_id=63f8822cc1ff4e4ca6a14ce9c4f9a39a, role_id=43106c664e1b40de984c0c62701149c8)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:59:22.313334 dv-cinder-master-3-smb devstack@keystone.service[80237]: WARNING py.warnings [None req-87466cec-5783-4adb-8fc5-6269ec6a9419 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 10 12:59:22.313334 dv-cinder-master-3-smb devstack@keystone.service[80237]: warnings.warn(msg) Jun 10 12:59:22.313334 dv-cinder-master-3-smb devstack@keystone.service[80237]: Jun 10 12:59:22.315203 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87466cec-5783-4adb-8fc5-6269ec6a9419 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:59:22.401265 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 754/1501] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 12:59:22 2022] PUT /identity/v3/projects/4b63dd2efe784b06a622cb89e6550327/users/63f8822cc1ff4e4ca6a14ce9c4f9a39a/roles/43106c664e1b40de984c0c62701149c8 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 12:59:22.412726 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591748ad-9c62-471f-b689-8c2c7235e980 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:59:22.413019 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591748ad-9c62-471f-b689-8c2c7235e980 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:59:22.413302 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591748ad-9c62-471f-b689-8c2c7235e980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:59:22.439045 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-591748ad-9c62-471f-b689-8c2c7235e980 None None] MFA Rules not processed for user `63f8822cc1ff4e4ca6a14ce9c4f9a39a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:59:22.440296 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-591748ad-9c62-471f-b689-8c2c7235e980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:59:22.505122 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 748/1502] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 12:59:22 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 12:59:22.519657 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1415cae4-57e4-4618-a1ba-7e54bf43d32f None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:59:22.520098 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1415cae4-57e4-4618-a1ba-7e54bf43d32f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:59:22.520480 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1415cae4-57e4-4618-a1ba-7e54bf43d32f None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:59:22.522490 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 755/1503] 10.222.0.27 () {54 vars in 963 bytes} [Fri Jun 10 12:59:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:59:22.528063 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df38742b-1c3b-4698-9744-518dd2125d7e None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:59:22.528386 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df38742b-1c3b-4698-9744-518dd2125d7e None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:59:22.528676 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df38742b-1c3b-4698-9744-518dd2125d7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:59:22.546515 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-df38742b-1c3b-4698-9744-518dd2125d7e None None] MFA Rules not processed for user `c6cc95a563b941ac92ab283aa0d5a5d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:59:22.547751 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-df38742b-1c3b-4698-9744-518dd2125d7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:59:22.590612 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:59:22.597696 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 749/1504] 10.222.0.27 () {58 vars in 1060 bytes} [Fri Jun 10 12:59:22 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 12:59:22.605270 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1415cae4-57e4-4618-a1ba-7e54bf43d32f None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:59:22.615987 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1415cae4-57e4-4618-a1ba-7e54bf43d32f None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:59:22.619516 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a320cd1b-4095-402e-bf30-51b6aeea7e74 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6cc95a563b941ac92ab283aa0d5a5d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:59:22.620179 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a320cd1b-4095-402e-bf30-51b6aeea7e74 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:59:22.620460 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a320cd1b-4095-402e-bf30-51b6aeea7e74 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:59:22.620736 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a320cd1b-4095-402e-bf30-51b6aeea7e74 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:59:22.628890 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a320cd1b-4095-402e-bf30-51b6aeea7e74 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:59:22.630284 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a320cd1b-4095-402e-bf30-51b6aeea7e74 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:59:22.639250 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 756/1505] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 12:59:22 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 10 12:59:22.680579 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9396bf36-267a-4778-8063-ff0b7e09937f None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:59:22.680875 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9396bf36-267a-4778-8063-ff0b7e09937f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:59:22.681160 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9396bf36-267a-4778-8063-ff0b7e09937f None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:59:22.682475 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 750/1506] 10.222.0.27 () {52 vars in 829 bytes} [Fri Jun 10 12:59:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 12:59:22.686004 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a320cd1b-4095-402e-bf30-51b6aeea7e74 service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:59:22.745434 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a320cd1b-4095-402e-bf30-51b6aeea7e74 service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:59:22.749834 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f24b420-3279-4493-be52-0c646506a1a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:59:22.750691 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f24b420-3279-4493-be52-0c646506a1a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:59:22.750969 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f24b420-3279-4493-be52-0c646506a1a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:59:22.751243 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f24b420-3279-4493-be52-0c646506a1a0 admin admin] PATH_INFO: `/v3/users/63f8822cc1ff4e4ca6a14ce9c4f9a39a` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:59:22.752425 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f24b420-3279-4493-be52-0c646506a1a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=63f8822cc1ff4e4ca6a14ce9c4f9a39a)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:59:22.753424 dv-cinder-master-3-smb devstack@keystone.service[80237]: WARNING py.warnings [None req-1f24b420-3279-4493-be52-0c646506a1a0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 10 12:59:22.753424 dv-cinder-master-3-smb devstack@keystone.service[80237]: warnings.warn(msg) Jun 10 12:59:22.753424 dv-cinder-master-3-smb devstack@keystone.service[80237]: Jun 10 12:59:22.754606 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f24b420-3279-4493-be52-0c646506a1a0 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:59:22.978557 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-1f24b420-3279-4493-be52-0c646506a1a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63f8822cc1ff4e4ca6a14ce9c4f9a39a'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 12:59:22.990627 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-1f24b420-3279-4493-be52-0c646506a1a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63f8822cc1ff4e4ca6a14ce9c4f9a39a'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 12:59:23.063995 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-1f24b420-3279-4493-be52-0c646506a1a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63f8822cc1ff4e4ca6a14ce9c4f9a39a'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 12:59:23.064696 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-1f24b420-3279-4493-be52-0c646506a1a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63f8822cc1ff4e4ca6a14ce9c4f9a39a'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 12:59:23.083156 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:59:23.085641 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-1f24b420-3279-4493-be52-0c646506a1a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63f8822cc1ff4e4ca6a14ce9c4f9a39a'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 12:59:23.091671 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 757/1507] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 12:59:22 2022] DELETE /identity/v3/users/63f8822cc1ff4e4ca6a14ce9c4f9a39a => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 12:59:23.096136 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affe989c-7820-44e5-91fc-fe6185a16f84 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:59:23.096640 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affe989c-7820-44e5-91fc-fe6185a16f84 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:59:23.096748 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affe989c-7820-44e5-91fc-fe6185a16f84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:59:23.115281 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-affe989c-7820-44e5-91fc-fe6185a16f84 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 12:59:23.116666 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-affe989c-7820-44e5-91fc-fe6185a16f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:59:23.174091 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 751/1508] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 12:59:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 12:59:23.183981 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f24b420-3279-4493-be52-0c646506a1a0 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:59:23.185395 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1f24b420-3279-4493-be52-0c646506a1a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:59:23.240948 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f24b420-3279-4493-be52-0c646506a1a0 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:59:23.245091 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17039e21-204e-4d3f-8e1a-2c46e2b92c0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:59:23.245735 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17039e21-204e-4d3f-8e1a-2c46e2b92c0b service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:59:23.246024 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17039e21-204e-4d3f-8e1a-2c46e2b92c0b service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:59:23.246307 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17039e21-204e-4d3f-8e1a-2c46e2b92c0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:59:23.255178 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17039e21-204e-4d3f-8e1a-2c46e2b92c0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:59:23.256635 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17039e21-204e-4d3f-8e1a-2c46e2b92c0b service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:59:23.265997 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 758/1509] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 12:59:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 12:59:23.737170 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affe989c-7820-44e5-91fc-fe6185a16f84 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 12:59:23.738598 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-affe989c-7820-44e5-91fc-fe6185a16f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 12:59:23.750449 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affe989c-7820-44e5-91fc-fe6185a16f84 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 12:59:23.755113 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90fc8ef-a7fc-4e2c-baaa-ba9909e888df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 12:59:23.756010 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90fc8ef-a7fc-4e2c-baaa-ba9909e888df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 12:59:23.756300 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90fc8ef-a7fc-4e2c-baaa-ba9909e888df admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 12:59:23.756581 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90fc8ef-a7fc-4e2c-baaa-ba9909e888df admin admin] PATH_INFO: `/v3/projects/4b63dd2efe784b06a622cb89e6550327` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 12:59:23.757902 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90fc8ef-a7fc-4e2c-baaa-ba9909e888df admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b63dd2efe784b06a622cb89e6550327)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 12:59:23.758926 dv-cinder-master-3-smb devstack@keystone.service[80238]: WARNING py.warnings [None req-a90fc8ef-a7fc-4e2c-baaa-ba9909e888df admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 10 12:59:23.758926 dv-cinder-master-3-smb devstack@keystone.service[80238]: warnings.warn(msg) Jun 10 12:59:23.758926 dv-cinder-master-3-smb devstack@keystone.service[80238]: Jun 10 12:59:23.760136 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90fc8ef-a7fc-4e2c-baaa-ba9909e888df admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 12:59:23.891466 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-a90fc8ef-a7fc-4e2c-baaa-ba9909e888df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b63dd2efe784b06a622cb89e6550327', 'request_id': 'req-a90fc8ef-a7fc-4e2c-baaa-ba9909e888df'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 12:59:23.897766 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-a90fc8ef-a7fc-4e2c-baaa-ba9909e888df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b63dd2efe784b06a622cb89e6550327', 'request_id': 'req-a90fc8ef-a7fc-4e2c-baaa-ba9909e888df'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 12:59:23.912062 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-a90fc8ef-a7fc-4e2c-baaa-ba9909e888df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b63dd2efe784b06a622cb89e6550327', 'request_id': 'req-a90fc8ef-a7fc-4e2c-baaa-ba9909e888df'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 12:59:23.989453 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-a90fc8ef-a7fc-4e2c-baaa-ba9909e888df admin admin] The token cache is being invalidate because project 4b63dd2efe784b06a622cb89e6550327 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 12:59:23.989749 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-a90fc8ef-a7fc-4e2c-baaa-ba9909e888df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 12:59:23.990494 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-a90fc8ef-a7fc-4e2c-baaa-ba9909e888df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 12:59:23.992159 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 752/1510] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 12:59:23 2022] DELETE /identity/v3/projects/4b63dd2efe784b06a622cb89e6550327 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 12:59:33.261880 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 12:59:33.986132 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:01:08.221364 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8787f3-f059-4145-8c87-f81d8cf991a9 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:01:08.221905 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8787f3-f059-4145-8c87-f81d8cf991a9 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:01:08.221905 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8787f3-f059-4145-8c87-f81d8cf991a9 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:01:08.223258 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 759/1511] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 10 13:01:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 13:01:08.228187 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74c159c-c671-494f-a4a7-1f4f3793e2de None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:01:08.228626 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74c159c-c671-494f-a4a7-1f4f3793e2de None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:01:08.228737 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74c159c-c671-494f-a4a7-1f4f3793e2de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:01:08.247026 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-d74c159c-c671-494f-a4a7-1f4f3793e2de None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:01:08.248265 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d74c159c-c671-494f-a4a7-1f4f3793e2de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:01:08.307593 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 753/1512] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 10 13:01:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:01:08.478666 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8787f3-f059-4145-8c87-f81d8cf991a9 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:01:08.480474 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-eb8787f3-f059-4145-8c87-f81d8cf991a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:01:08.554571 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8787f3-f059-4145-8c87-f81d8cf991a9 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:01:08.559940 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6210d3eb-dc7c-4427-8c98-08f1d42e317c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bd6caa461e84100a3d5fe5af5d80f3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:01:08.560584 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6210d3eb-dc7c-4427-8c98-08f1d42e317c service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:01:08.560870 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6210d3eb-dc7c-4427-8c98-08f1d42e317c service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:01:08.561149 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6210d3eb-dc7c-4427-8c98-08f1d42e317c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:01:08.569226 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6210d3eb-dc7c-4427-8c98-08f1d42e317c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:01:08.570799 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6210d3eb-dc7c-4427-8c98-08f1d42e317c service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:01:08.580174 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 760/1513] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:01:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:01:09.443507 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deefc673-a256-4ece-92b2-8f11d1eada0d None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:01:09.444074 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deefc673-a256-4ece-92b2-8f11d1eada0d None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:01:09.444074 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deefc673-a256-4ece-92b2-8f11d1eada0d None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:01:09.445538 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 754/1514] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 10 13:01:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 13:01:09.450371 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84704240-0e0a-4c6a-b041-013fa7b6fa1d None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:01:09.450820 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84704240-0e0a-4c6a-b041-013fa7b6fa1d None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:01:09.450926 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84704240-0e0a-4c6a-b041-013fa7b6fa1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:01:09.468956 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-84704240-0e0a-4c6a-b041-013fa7b6fa1d None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:01:09.470156 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-84704240-0e0a-4c6a-b041-013fa7b6fa1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:01:09.477873 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 761/1515] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 10 13:01:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:01:09.629632 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deefc673-a256-4ece-92b2-8f11d1eada0d None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:01:09.643602 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deefc673-a256-4ece-92b2-8f11d1eada0d None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:01:09.648925 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3119c805-9721-456c-be6f-6dc466ceb377 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bd6caa461e84100a3d5fe5af5d80f3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:01:09.649556 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3119c805-9721-456c-be6f-6dc466ceb377 service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:01:09.649840 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3119c805-9721-456c-be6f-6dc466ceb377 service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:01:09.650119 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3119c805-9721-456c-be6f-6dc466ceb377 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:01:09.658323 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3119c805-9721-456c-be6f-6dc466ceb377 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:01:09.659953 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3119c805-9721-456c-be6f-6dc466ceb377 service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:01:09.669256 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 755/1516] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:01:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:01:10.121330 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500ed17c-2e5a-464d-a35b-1c2f8f963df2 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:01:10.121621 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500ed17c-2e5a-464d-a35b-1c2f8f963df2 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:01:10.121900 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500ed17c-2e5a-464d-a35b-1c2f8f963df2 None None] PATH_INFO: `/v3` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:01:10.123200 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 762/1517] 10.222.0.36 () {54 vars in 925 bytes} [Fri Jun 10 13:01:10 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 13:01:10.129102 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406c30c7-f5b9-492d-bb33-ddffdc2af06e None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:01:10.129396 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406c30c7-f5b9-492d-bb33-ddffdc2af06e None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:01:10.129681 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406c30c7-f5b9-492d-bb33-ddffdc2af06e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:01:10.147299 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-406c30c7-f5b9-492d-bb33-ddffdc2af06e None None] MFA Rules not processed for user `d972ab782b3f4d2aba575314f16dafc7`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:01:10.148528 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-406c30c7-f5b9-492d-bb33-ddffdc2af06e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:01:10.209459 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 756/1518] 10.222.0.36 () {58 vars in 1015 bytes} [Fri Jun 10 13:01:10 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:01:19.343613 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c007d43e-5ea0-4624-8bc8-2745ca5ad50e None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:01:19.344238 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c007d43e-5ea0-4624-8bc8-2745ca5ad50e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:01:19.344238 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c007d43e-5ea0-4624-8bc8-2745ca5ad50e None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:01:19.345595 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 763/1519] 10.222.0.27 () {54 vars in 960 bytes} [Fri Jun 10 13:01:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 13:01:19.350578 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852aee88-642a-482b-a2fc-0c93ef3ab8cc None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:01:19.351010 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852aee88-642a-482b-a2fc-0c93ef3ab8cc None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:01:19.351146 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852aee88-642a-482b-a2fc-0c93ef3ab8cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:01:19.369467 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-852aee88-642a-482b-a2fc-0c93ef3ab8cc None None] MFA Rules not processed for user `d972ab782b3f4d2aba575314f16dafc7`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:01:19.370667 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-852aee88-642a-482b-a2fc-0c93ef3ab8cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:01:19.379793 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 757/1520] 10.222.0.27 () {58 vars in 1057 bytes} [Fri Jun 10 13:01:19 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:01:19.387068 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c007d43e-5ea0-4624-8bc8-2745ca5ad50e None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:01:19.400175 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c007d43e-5ea0-4624-8bc8-2745ca5ad50e None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:01:19.405346 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016ea0d0-b29c-4663-b082-4871d0aace87 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd972ab782b3f4d2aba575314f16dafc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:01:19.405972 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016ea0d0-b29c-4663-b082-4871d0aace87 service placement] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:01:19.406258 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016ea0d0-b29c-4663-b082-4871d0aace87 service placement] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:01:19.406540 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016ea0d0-b29c-4663-b082-4871d0aace87 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:01:19.414728 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016ea0d0-b29c-4663-b082-4871d0aace87 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:01:19.416321 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016ea0d0-b29c-4663-b082-4871d0aace87 service placement] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:01:19.426333 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 764/1521] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:01:19 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:01:20.935038 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97915eb0-e460-4f76-be15-26eeef79b393 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:01:20.935601 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97915eb0-e460-4f76-be15-26eeef79b393 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:01:20.935601 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97915eb0-e460-4f76-be15-26eeef79b393 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:01:20.937021 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 758/1522] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 10 13:01:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 13:01:20.942424 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6418a535-6622-4788-8a7d-c4c624869918 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:01:20.942841 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6418a535-6622-4788-8a7d-c4c624869918 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:01:20.942969 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6418a535-6622-4788-8a7d-c4c624869918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:01:20.960516 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-6418a535-6622-4788-8a7d-c4c624869918 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:01:20.961710 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6418a535-6622-4788-8a7d-c4c624869918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:01:20.969474 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 765/1523] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 10 13:01:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:01:21.121491 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97915eb0-e460-4f76-be15-26eeef79b393 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:01:21.136218 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97915eb0-e460-4f76-be15-26eeef79b393 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:01:21.141535 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34011b59-638c-43d0-9009-0518d5ae17c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bd6caa461e84100a3d5fe5af5d80f3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:01:21.142165 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34011b59-638c-43d0-9009-0518d5ae17c1 service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:01:21.142449 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34011b59-638c-43d0-9009-0518d5ae17c1 service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:01:21.142727 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34011b59-638c-43d0-9009-0518d5ae17c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:01:21.150763 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34011b59-638c-43d0-9009-0518d5ae17c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:01:21.156557 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34011b59-638c-43d0-9009-0518d5ae17c1 service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:01:21.161947 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 759/1524] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:01:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:01:22.033182 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c01b069-2e42-435e-939e-7ab30443edbe None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:01:22.033740 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c01b069-2e42-435e-939e-7ab30443edbe None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:01:22.033740 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c01b069-2e42-435e-939e-7ab30443edbe None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:01:22.035631 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 766/1525] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 10 13:01:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 13:01:22.040467 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af5efdf-ef0f-4da3-962e-c94d358026ef None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:01:22.044387 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af5efdf-ef0f-4da3-962e-c94d358026ef None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:01:22.044387 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af5efdf-ef0f-4da3-962e-c94d358026ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:01:22.059322 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-9af5efdf-ef0f-4da3-962e-c94d358026ef None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:01:22.060550 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-9af5efdf-ef0f-4da3-962e-c94d358026ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:01:22.068403 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 760/1526] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 10 13:01:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:01:22.219265 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c01b069-2e42-435e-939e-7ab30443edbe None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:01:22.233839 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c01b069-2e42-435e-939e-7ab30443edbe None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:01:22.239757 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d686f4c9-e7cf-46bf-bde3-7e74638988b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bd6caa461e84100a3d5fe5af5d80f3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:01:22.240411 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d686f4c9-e7cf-46bf-bde3-7e74638988b4 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:01:22.240693 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d686f4c9-e7cf-46bf-bde3-7e74638988b4 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:01:22.240967 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d686f4c9-e7cf-46bf-bde3-7e74638988b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:01:22.248915 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d686f4c9-e7cf-46bf-bde3-7e74638988b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:01:22.250477 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d686f4c9-e7cf-46bf-bde3-7e74638988b4 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:01:22.259646 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 767/1527] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:01:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:01:26.345793 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8d25ae-995f-483b-94e8-5b26d35b8203 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:01:26.346316 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8d25ae-995f-483b-94e8-5b26d35b8203 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:01:26.346316 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8d25ae-995f-483b-94e8-5b26d35b8203 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:01:26.348226 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 761/1528] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 10 13:01:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 13:01:26.353316 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c2a1f2-53e0-4691-909d-bc1bdeca5e73 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:01:26.353797 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c2a1f2-53e0-4691-909d-bc1bdeca5e73 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:01:26.353904 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c2a1f2-53e0-4691-909d-bc1bdeca5e73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:01:26.371306 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-19c2a1f2-53e0-4691-909d-bc1bdeca5e73 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:01:26.372511 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-19c2a1f2-53e0-4691-909d-bc1bdeca5e73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:01:26.380134 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 768/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 10 13:01:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:01:26.418876 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8d25ae-995f-483b-94e8-5b26d35b8203 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:01:26.420526 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-bc8d25ae-995f-483b-94e8-5b26d35b8203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:01:26.477055 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8d25ae-995f-483b-94e8-5b26d35b8203 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:01:26.480506 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bc71f4-60ad-42f0-9e82-da4da4867d05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:01:26.481135 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bc71f4-60ad-42f0-9e82-da4da4867d05 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:01:26.481413 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bc71f4-60ad-42f0-9e82-da4da4867d05 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:01:26.481686 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bc71f4-60ad-42f0-9e82-da4da4867d05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:01:26.490352 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bc71f4-60ad-42f0-9e82-da4da4867d05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:01:26.492063 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bc71f4-60ad-42f0-9e82-da4da4867d05 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:01:26.496908 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:01:26.501409 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 762/1530] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:01:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:01:28.602453 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1d518b-52e1-4f64-914f-da599fd32915 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:01:28.602932 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1d518b-52e1-4f64-914f-da599fd32915 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:01:28.603059 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1d518b-52e1-4f64-914f-da599fd32915 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:01:28.604518 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 769/1531] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 10 13:01:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 13:01:28.609912 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f94364-2972-488c-991e-ea455367213d None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:01:28.610332 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f94364-2972-488c-991e-ea455367213d None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:01:28.610498 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f94364-2972-488c-991e-ea455367213d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:01:28.629687 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-10f94364-2972-488c-991e-ea455367213d None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:01:28.630872 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-10f94364-2972-488c-991e-ea455367213d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:01:28.638495 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 763/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 10 13:01:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:01:28.674540 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1d518b-52e1-4f64-914f-da599fd32915 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:01:28.685035 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1d518b-52e1-4f64-914f-da599fd32915 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:01:28.688480 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6d7a1a-9cb8-4053-9fa7-ec099e1f54f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:01:28.689100 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6d7a1a-9cb8-4053-9fa7-ec099e1f54f4 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:01:28.689387 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6d7a1a-9cb8-4053-9fa7-ec099e1f54f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:01:28.689660 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6d7a1a-9cb8-4053-9fa7-ec099e1f54f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:01:28.699060 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6d7a1a-9cb8-4053-9fa7-ec099e1f54f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:01:28.700538 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6d7a1a-9cb8-4053-9fa7-ec099e1f54f4 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:01:28.710339 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 770/1533] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:01:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:01:32.268802 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba11896a-6cf8-4fc1-ab9d-ea5139f5ad84 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:01:32.269337 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba11896a-6cf8-4fc1-ab9d-ea5139f5ad84 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:01:32.269337 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba11896a-6cf8-4fc1-ab9d-ea5139f5ad84 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:01:32.270672 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 764/1534] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 10 13:01:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 13:01:32.275745 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1dc04d-33fb-45f2-b8c9-45514bde1c31 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:01:32.276204 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1dc04d-33fb-45f2-b8c9-45514bde1c31 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:01:32.276315 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1dc04d-33fb-45f2-b8c9-45514bde1c31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:01:32.293963 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-be1dc04d-33fb-45f2-b8c9-45514bde1c31 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:01:32.295133 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-be1dc04d-33fb-45f2-b8c9-45514bde1c31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:01:32.303266 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 771/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 10 13:01:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:01:32.592903 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba11896a-6cf8-4fc1-ab9d-ea5139f5ad84 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:01:32.594551 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-ba11896a-6cf8-4fc1-ab9d-ea5139f5ad84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:01:32.650288 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba11896a-6cf8-4fc1-ab9d-ea5139f5ad84 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:01:32.654124 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a439817f-e9a4-4a85-a794-63bccf11ef61 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6cc95a563b941ac92ab283aa0d5a5d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:01:32.654788 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a439817f-e9a4-4a85-a794-63bccf11ef61 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:01:32.655080 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a439817f-e9a4-4a85-a794-63bccf11ef61 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:01:32.655356 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a439817f-e9a4-4a85-a794-63bccf11ef61 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:01:32.664642 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a439817f-e9a4-4a85-a794-63bccf11ef61 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:01:32.666075 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a439817f-e9a4-4a85-a794-63bccf11ef61 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:01:32.675983 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 765/1536] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:01:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:01:34.798610 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a32235a-609b-4d80-b148-2a3b9d863c09 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:01:34.799189 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a32235a-609b-4d80-b148-2a3b9d863c09 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:01:34.799297 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a32235a-609b-4d80-b148-2a3b9d863c09 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:01:34.800612 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 772/1537] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 10 13:01:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 13:01:34.805941 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48802ad1-e0ce-4b79-b1b9-ad4638ab9efe None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:01:34.806359 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48802ad1-e0ce-4b79-b1b9-ad4638ab9efe None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:01:34.806538 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48802ad1-e0ce-4b79-b1b9-ad4638ab9efe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:01:34.825171 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-48802ad1-e0ce-4b79-b1b9-ad4638ab9efe None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:01:34.826366 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-48802ad1-e0ce-4b79-b1b9-ad4638ab9efe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:01:34.835003 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 766/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 10 13:01:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:01:35.098658 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a32235a-609b-4d80-b148-2a3b9d863c09 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:01:35.100386 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9a32235a-609b-4d80-b148-2a3b9d863c09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:01:35.107892 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=65 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:01:35.114110 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a32235a-609b-4d80-b148-2a3b9d863c09 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:01:35.117558 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa054ab8-5e7a-4df3-9b3c-789793267fa7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6cc95a563b941ac92ab283aa0d5a5d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:01:35.118248 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa054ab8-5e7a-4df3-9b3c-789793267fa7 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:01:35.118530 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa054ab8-5e7a-4df3-9b3c-789793267fa7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:01:35.122317 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa054ab8-5e7a-4df3-9b3c-789793267fa7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:01:35.128540 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa054ab8-5e7a-4df3-9b3c-789793267fa7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:01:35.129947 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa054ab8-5e7a-4df3-9b3c-789793267fa7 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:01:35.139183 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 773/1539] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:01:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:01:44.819762 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=32 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:01:45.141260 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=5 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:03:15.496678 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d0c6be-2c1a-418e-85c2-d3156b79bfc1 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:15.516064 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d0c6be-2c1a-418e-85c2-d3156b79bfc1 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:15.516064 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d0c6be-2c1a-418e-85c2-d3156b79bfc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:15.537095 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-91d0c6be-2c1a-418e-85c2-d3156b79bfc1 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:03:15.538339 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-91d0c6be-2c1a-418e-85c2-d3156b79bfc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:15.547231 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 767/1540] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:03:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 10 13:03:15.565371 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006be520-8407-40fa-b66d-f3b9a23d4a61 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:15.566086 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006be520-8407-40fa-b66d-f3b9a23d4a61 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:15.566086 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006be520-8407-40fa-b66d-f3b9a23d4a61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:15.604505 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-006be520-8407-40fa-b66d-f3b9a23d4a61 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:03:15.605777 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-006be520-8407-40fa-b66d-f3b9a23d4a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:15.617659 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 774/1541] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:03:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:03:15.623146 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d0c6be-2c1a-418e-85c2-d3156b79bfc1 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:15.633050 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc248e9-e508-4606-aed7-15ee50583bf5 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:15.633399 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc248e9-e508-4606-aed7-15ee50583bf5 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:15.634049 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc248e9-e508-4606-aed7-15ee50583bf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:15.638067 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d0c6be-2c1a-418e-85c2-d3156b79bfc1 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:15.654977 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06984edc-8da9-449a-a59d-a838548da3ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:15.655687 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06984edc-8da9-449a-a59d-a838548da3ed admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:15.656004 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06984edc-8da9-449a-a59d-a838548da3ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:15.656276 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06984edc-8da9-449a-a59d-a838548da3ed admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:15.657028 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06984edc-8da9-449a-a59d-a838548da3ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 13:03:15.657335 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06984edc-8da9-449a-a59d-a838548da3ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:15.659023 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06984edc-8da9-449a-a59d-a838548da3ed admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:15.661518 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-2cc248e9-e508-4606-aed7-15ee50583bf5 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:03:15.663148 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2cc248e9-e508-4606-aed7-15ee50583bf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:15.678435 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 775/1542] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:03:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 10 13:03:15.682429 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 768/1543] 10.222.0.27 () {58 vars in 1152 bytes} [Fri Jun 10 13:03:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 13:03:15.690806 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84988e8f-1379-471c-a75d-d1bebb1090f0 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:15.691279 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06984edc-8da9-449a-a59d-a838548da3ed admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:15.691792 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84988e8f-1379-471c-a75d-d1bebb1090f0 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:15.692484 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84988e8f-1379-471c-a75d-d1bebb1090f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:15.704812 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06984edc-8da9-449a-a59d-a838548da3ed admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:15.725653 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-84988e8f-1379-471c-a75d-d1bebb1090f0 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:03:15.727440 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-84988e8f-1379-471c-a75d-d1bebb1090f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:15.728694 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734a0069-2d0b-4b99-aade-e041462f67b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:15.729352 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734a0069-2d0b-4b99-aade-e041462f67b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:15.729633 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734a0069-2d0b-4b99-aade-e041462f67b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:15.729907 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734a0069-2d0b-4b99-aade-e041462f67b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:15.730602 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734a0069-2d0b-4b99-aade-e041462f67b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:15.735906 dv-cinder-master-3-smb devstack@keystone.service[80238]: WARNING py.warnings [None req-734a0069-2d0b-4b99-aade-e041462f67b1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 10 13:03:15.735906 dv-cinder-master-3-smb devstack@keystone.service[80238]: warnings.warn(msg) Jun 10 13:03:15.735906 dv-cinder-master-3-smb devstack@keystone.service[80238]: Jun 10 13:03:15.735906 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734a0069-2d0b-4b99-aade-e041462f67b1 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:15.744864 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 776/1544] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:03:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 10 13:03:15.751444 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84988e8f-1379-471c-a75d-d1bebb1090f0 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:15.765433 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84988e8f-1379-471c-a75d-d1bebb1090f0 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:15.777930 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8144ed3f-4c63-4de5-a469-d46486cc996d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:15.779346 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8144ed3f-4c63-4de5-a469-d46486cc996d admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:15.779714 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8144ed3f-4c63-4de5-a469-d46486cc996d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:15.780290 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8144ed3f-4c63-4de5-a469-d46486cc996d admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:15.781356 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8144ed3f-4c63-4de5-a469-d46486cc996d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 13:03:15.781954 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8144ed3f-4c63-4de5-a469-d46486cc996d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:15.783259 dv-cinder-master-3-smb devstack@keystone.service[80237]: WARNING py.warnings [None req-8144ed3f-4c63-4de5-a469-d46486cc996d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 10 13:03:15.783259 dv-cinder-master-3-smb devstack@keystone.service[80237]: warnings.warn(msg) Jun 10 13:03:15.783259 dv-cinder-master-3-smb devstack@keystone.service[80237]: Jun 10 13:03:15.784634 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8144ed3f-4c63-4de5-a469-d46486cc996d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:15.803703 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 777/1545] 10.222.0.27 () {58 vars in 1152 bytes} [Fri Jun 10 13:03:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 13:03:15.813239 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8144ed3f-4c63-4de5-a469-d46486cc996d admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:15.825493 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8144ed3f-4c63-4de5-a469-d46486cc996d admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:15.837269 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed06840-dea8-4a13-adfe-89bb20b1b074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:15.838203 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed06840-dea8-4a13-adfe-89bb20b1b074 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:15.838802 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed06840-dea8-4a13-adfe-89bb20b1b074 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:15.839314 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed06840-dea8-4a13-adfe-89bb20b1b074 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:15.840339 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed06840-dea8-4a13-adfe-89bb20b1b074 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:15.842518 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed06840-dea8-4a13-adfe-89bb20b1b074 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:15.872075 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 769/1546] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:03:15 2022] POST /identity/v3/projects => generated 377 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 10 13:03:15.874157 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3245c622-7144-4e45-ac8c-979e73220dd0 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:15.874457 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3245c622-7144-4e45-ac8c-979e73220dd0 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:15.874738 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3245c622-7144-4e45-ac8c-979e73220dd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:15.894226 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-3245c622-7144-4e45-ac8c-979e73220dd0 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:03:15.900872 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-3245c622-7144-4e45-ac8c-979e73220dd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:15.929426 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 778/1547] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:03:15 2022] POST /identity/v3/projects => generated 377 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 10 13:03:15.935469 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f452d2-62cd-45da-b283-d7a3aed9e210 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:15.935469 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f452d2-62cd-45da-b283-d7a3aed9e210 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:15.935469 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f452d2-62cd-45da-b283-d7a3aed9e210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:15.951056 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-e5f452d2-62cd-45da-b283-d7a3aed9e210 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:03:15.952296 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e5f452d2-62cd-45da-b283-d7a3aed9e210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:15.962935 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 770/1548] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:03:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:03:15.964220 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150f1e36-fbe0-4292-a4a7-a43d3689e376 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:15.964515 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150f1e36-fbe0-4292-a4a7-a43d3689e376 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:15.964803 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150f1e36-fbe0-4292-a4a7-a43d3689e376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:15.986478 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-150f1e36-fbe0-4292-a4a7-a43d3689e376 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:03:15.992301 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-150f1e36-fbe0-4292-a4a7-a43d3689e376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:16.015336 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 779/1549] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:03:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:03:16.015933 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f452d2-62cd-45da-b283-d7a3aed9e210 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:16.024747 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f452d2-62cd-45da-b283-d7a3aed9e210 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:16.030945 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17d6a55-f4eb-4ca7-ba69-a4a07ebdce95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:16.031615 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17d6a55-f4eb-4ca7-ba69-a4a07ebdce95 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:16.031930 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17d6a55-f4eb-4ca7-ba69-a4a07ebdce95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:16.032920 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17d6a55-f4eb-4ca7-ba69-a4a07ebdce95 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:16.032920 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17d6a55-f4eb-4ca7-ba69-a4a07ebdce95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:16.033960 dv-cinder-master-3-smb devstack@keystone.service[80237]: WARNING py.warnings [None req-e17d6a55-f4eb-4ca7-ba69-a4a07ebdce95 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 10 13:03:16.033960 dv-cinder-master-3-smb devstack@keystone.service[80237]: warnings.warn(msg) Jun 10 13:03:16.033960 dv-cinder-master-3-smb devstack@keystone.service[80237]: Jun 10 13:03:16.035323 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17d6a55-f4eb-4ca7-ba69-a4a07ebdce95 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:16.058481 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 771/1550] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:03:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:03:16.060949 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d63807a-0b11-4ceb-a9ba-ca0d4c39217c None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:16.060949 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d63807a-0b11-4ceb-a9ba-ca0d4c39217c None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:16.060949 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d63807a-0b11-4ceb-a9ba-ca0d4c39217c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:16.077689 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-1d63807a-0b11-4ceb-a9ba-ca0d4c39217c None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:03:16.078867 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-1d63807a-0b11-4ceb-a9ba-ca0d4c39217c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:16.086638 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 772/1551] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:03:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:03:16.087815 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9262f310-c3cf-43ee-b034-e78d4023740f None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:16.088135 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9262f310-c3cf-43ee-b034-e78d4023740f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:16.088415 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9262f310-c3cf-43ee-b034-e78d4023740f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:16.105933 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-9262f310-c3cf-43ee-b034-e78d4023740f None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:03:16.107725 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-9262f310-c3cf-43ee-b034-e78d4023740f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:16.115306 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 773/1552] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:03:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:03:16.116062 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9262f310-c3cf-43ee-b034-e78d4023740f None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:16.124997 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9262f310-c3cf-43ee-b034-e78d4023740f None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:16.129511 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0cfa98-f7ab-4176-9ea7-3be40e50ed1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:16.130150 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0cfa98-f7ab-4176-9ea7-3be40e50ed1c admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:16.130434 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0cfa98-f7ab-4176-9ea7-3be40e50ed1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:16.130712 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0cfa98-f7ab-4176-9ea7-3be40e50ed1c admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:16.131385 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0cfa98-f7ab-4176-9ea7-3be40e50ed1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:16.133331 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0cfa98-f7ab-4176-9ea7-3be40e50ed1c admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:16.167737 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 780/1553] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:03:16 2022] POST /identity/v3/users => generated 289 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:03:16.168871 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26cf0e3-dbdc-4171-9921-ed7782f1a10c None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:16.169743 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26cf0e3-dbdc-4171-9921-ed7782f1a10c None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:16.170025 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26cf0e3-dbdc-4171-9921-ed7782f1a10c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:16.188201 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-e26cf0e3-dbdc-4171-9921-ed7782f1a10c None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:03:16.189385 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e26cf0e3-dbdc-4171-9921-ed7782f1a10c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:16.197045 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 781/1554] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:03:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:03:16.198191 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aac31f4-1995-4a37-a8ad-99e11e40b041 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:16.198480 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aac31f4-1995-4a37-a8ad-99e11e40b041 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:16.198759 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aac31f4-1995-4a37-a8ad-99e11e40b041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:16.215787 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-4aac31f4-1995-4a37-a8ad-99e11e40b041 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:03:16.217472 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-4aac31f4-1995-4a37-a8ad-99e11e40b041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:16.225099 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 782/1555] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:03:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:03:16.226267 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fa5446-dfc8-48d8-88e2-0731ec9ca4e3 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:16.226548 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fa5446-dfc8-48d8-88e2-0731ec9ca4e3 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:16.226827 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fa5446-dfc8-48d8-88e2-0731ec9ca4e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:16.242350 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 774/1556] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:03:16 2022] POST /identity/v3/users => generated 289 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:03:16.243981 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406c8459-2b18-426d-8f96-bd2a35a23dba None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:16.244101 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-c7fa5446-dfc8-48d8-88e2-0731ec9ca4e3 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:03:16.244382 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406c8459-2b18-426d-8f96-bd2a35a23dba None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:16.244725 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406c8459-2b18-426d-8f96-bd2a35a23dba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:16.245180 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c7fa5446-dfc8-48d8-88e2-0731ec9ca4e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:16.253017 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 783/1557] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:03:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:03:16.254216 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2269b260-f616-4f63-b3a7-c267af64a9c0 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:16.254508 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2269b260-f616-4f63-b3a7-c267af64a9c0 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:16.254788 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2269b260-f616-4f63-b3a7-c267af64a9c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:16.263662 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-406c8459-2b18-426d-8f96-bd2a35a23dba None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:03:16.264877 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-406c8459-2b18-426d-8f96-bd2a35a23dba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:16.271981 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-2269b260-f616-4f63-b3a7-c267af64a9c0 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:03:16.272577 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 775/1558] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:03:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:03:16.273089 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2269b260-f616-4f63-b3a7-c267af64a9c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:16.277630 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b5fdda-347f-445a-a806-f54a291f3edd None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:16.277919 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b5fdda-347f-445a-a806-f54a291f3edd None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:16.278198 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b5fdda-347f-445a-a806-f54a291f3edd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:16.281609 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 784/1559] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:03:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:03:16.282370 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2269b260-f616-4f63-b3a7-c267af64a9c0 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:16.291417 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2269b260-f616-4f63-b3a7-c267af64a9c0 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:16.296484 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e10b808-a20f-4fb4-a6d3-40786ab171a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:16.296973 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-b4b5fdda-347f-445a-a806-f54a291f3edd None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:03:16.297115 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e10b808-a20f-4fb4-a6d3-40786ab171a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:16.297402 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e10b808-a20f-4fb4-a6d3-40786ab171a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:16.297680 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e10b808-a20f-4fb4-a6d3-40786ab171a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:16.298118 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b4b5fdda-347f-445a-a806-f54a291f3edd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:16.298426 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e10b808-a20f-4fb4-a6d3-40786ab171a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:16.300037 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e10b808-a20f-4fb4-a6d3-40786ab171a9 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:16.305907 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 776/1560] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:03:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:03:16.307189 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b76075-8bcc-46a8-985e-fa50ed531a81 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:16.307439 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b76075-8bcc-46a8-985e-fa50ed531a81 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:16.307721 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b76075-8bcc-46a8-985e-fa50ed531a81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:16.310235 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 785/1561] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:03:16 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:03:16.310915 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e10b808-a20f-4fb4-a6d3-40786ab171a9 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:16.323624 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e10b808-a20f-4fb4-a6d3-40786ab171a9 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:16.328817 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c2cb51-9e9b-4873-a565-fb4feb1655fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:16.329461 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c2cb51-9e9b-4873-a565-fb4feb1655fd admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:16.329750 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c2cb51-9e9b-4873-a565-fb4feb1655fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:16.330032 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c2cb51-9e9b-4873-a565-fb4feb1655fd admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:16.330752 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c2cb51-9e9b-4873-a565-fb4feb1655fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 13:03:16.331060 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c2cb51-9e9b-4873-a565-fb4feb1655fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:16.332169 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-57b76075-8bcc-46a8-985e-fa50ed531a81 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:03:16.332671 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c2cb51-9e9b-4873-a565-fb4feb1655fd admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:16.333373 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-57b76075-8bcc-46a8-985e-fa50ed531a81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:16.342028 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 777/1562] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:03:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:03:16.343025 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b76075-8bcc-46a8-985e-fa50ed531a81 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:16.352741 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 786/1563] 10.222.0.27 () {58 vars in 1152 bytes} [Fri Jun 10 13:03:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 13:03:16.352741 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c2cb51-9e9b-4873-a565-fb4feb1655fd admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:16.353017 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b76075-8bcc-46a8-985e-fa50ed531a81 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:16.357594 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8df4fa-798d-43d4-9a63-a8eee265e956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:16.358263 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c2cb51-9e9b-4873-a565-fb4feb1655fd admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:16.358383 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8df4fa-798d-43d4-9a63-a8eee265e956 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:16.358556 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8df4fa-798d-43d4-9a63-a8eee265e956 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:16.358841 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8df4fa-798d-43d4-9a63-a8eee265e956 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:16.359567 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8df4fa-798d-43d4-9a63-a8eee265e956 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:16.361209 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8df4fa-798d-43d4-9a63-a8eee265e956 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:16.363105 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05106d08-61f2-4dc0-bd00-e2a662b50863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:16.363371 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:03:16.364191 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05106d08-61f2-4dc0-bd00-e2a662b50863 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:16.364419 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05106d08-61f2-4dc0-bd00-e2a662b50863 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:16.364730 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05106d08-61f2-4dc0-bd00-e2a662b50863 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:16.365453 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05106d08-61f2-4dc0-bd00-e2a662b50863 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 13:03:16.365762 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05106d08-61f2-4dc0-bd00-e2a662b50863 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:16.367331 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05106d08-61f2-4dc0-bd00-e2a662b50863 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:16.371852 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 778/1564] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:03:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:03:16.372592 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8df4fa-798d-43d4-9a63-a8eee265e956 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:16.387198 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 787/1565] 10.222.0.27 () {58 vars in 1152 bytes} [Fri Jun 10 13:03:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 13:03:16.387198 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05106d08-61f2-4dc0-bd00-e2a662b50863 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:16.395004 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8df4fa-798d-43d4-9a63-a8eee265e956 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:16.395004 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aad4e1-d748-42e1-a568-80e494ab1add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:16.395004 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aad4e1-d748-42e1-a568-80e494ab1add admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:16.395004 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aad4e1-d748-42e1-a568-80e494ab1add admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:16.395389 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aad4e1-d748-42e1-a568-80e494ab1add admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:16.395389 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7aad4e1-d748-42e1-a568-80e494ab1add admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 13:03:16.395389 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7aad4e1-d748-42e1-a568-80e494ab1add admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:16.395389 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7aad4e1-d748-42e1-a568-80e494ab1add admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:16.397199 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05106d08-61f2-4dc0-bd00-e2a662b50863 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:16.402004 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b37827-2c5c-4ab4-a740-dc67d904cd19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:16.402646 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b37827-2c5c-4ab4-a740-dc67d904cd19 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:16.402931 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b37827-2c5c-4ab4-a740-dc67d904cd19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:16.403206 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b37827-2c5c-4ab4-a740-dc67d904cd19 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:16.404491 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b37827-2c5c-4ab4-a740-dc67d904cd19 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 13:03:16.404800 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b37827-2c5c-4ab4-a740-dc67d904cd19 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:16.406365 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b37827-2c5c-4ab4-a740-dc67d904cd19 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:16.408751 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 779/1566] 10.222.0.27 () {58 vars in 1152 bytes} [Fri Jun 10 13:03:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 13:03:16.409595 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aad4e1-d748-42e1-a568-80e494ab1add admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:16.421357 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aad4e1-d748-42e1-a568-80e494ab1add admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:16.425899 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 788/1567] 10.222.0.27 () {58 vars in 1152 bytes} [Fri Jun 10 13:03:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 13:03:16.428867 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b37827-2c5c-4ab4-a740-dc67d904cd19 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:16.429004 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feffa417-7fcc-4af5-9658-57ab8d1ea7e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:16.429004 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feffa417-7fcc-4af5-9658-57ab8d1ea7e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:16.429004 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feffa417-7fcc-4af5-9658-57ab8d1ea7e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:16.429004 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feffa417-7fcc-4af5-9658-57ab8d1ea7e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:16.433238 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feffa417-7fcc-4af5-9658-57ab8d1ea7e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 13:03:16.433743 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feffa417-7fcc-4af5-9658-57ab8d1ea7e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:16.435965 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feffa417-7fcc-4af5-9658-57ab8d1ea7e1 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:16.437870 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b37827-2c5c-4ab4-a740-dc67d904cd19 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:16.442919 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6077560c-36f5-4ad5-9c9e-942c6ba6cd61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:16.443583 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6077560c-36f5-4ad5-9c9e-942c6ba6cd61 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:16.443893 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6077560c-36f5-4ad5-9c9e-942c6ba6cd61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:16.444172 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6077560c-36f5-4ad5-9c9e-942c6ba6cd61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:16.444894 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6077560c-36f5-4ad5-9c9e-942c6ba6cd61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:16.446476 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6077560c-36f5-4ad5-9c9e-942c6ba6cd61 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:16.452532 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:03:16.453620 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 780/1568] 10.222.0.27 () {58 vars in 1152 bytes} [Fri Jun 10 13:03:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 13:03:16.454595 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feffa417-7fcc-4af5-9658-57ab8d1ea7e1 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:16.464331 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 789/1569] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:03:16 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:03:16.464736 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feffa417-7fcc-4af5-9658-57ab8d1ea7e1 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:16.465724 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6077560c-36f5-4ad5-9c9e-942c6ba6cd61 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:16.470489 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddfd920-87af-48e4-9a41-e96ae0bc975c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:16.471164 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddfd920-87af-48e4-9a41-e96ae0bc975c admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:16.471448 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddfd920-87af-48e4-9a41-e96ae0bc975c admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:16.471724 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddfd920-87af-48e4-9a41-e96ae0bc975c admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:16.472490 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddfd920-87af-48e4-9a41-e96ae0bc975c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 13:03:16.472793 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddfd920-87af-48e4-9a41-e96ae0bc975c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:16.474388 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddfd920-87af-48e4-9a41-e96ae0bc975c admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:16.477531 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6077560c-36f5-4ad5-9c9e-942c6ba6cd61 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:16.483155 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81c47c1-52d7-47a0-a923-8ebe16048341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:16.483935 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81c47c1-52d7-47a0-a923-8ebe16048341 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:16.484290 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81c47c1-52d7-47a0-a923-8ebe16048341 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:16.484638 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81c47c1-52d7-47a0-a923-8ebe16048341 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:16.485391 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81c47c1-52d7-47a0-a923-8ebe16048341 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:16.487453 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81c47c1-52d7-47a0-a923-8ebe16048341 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:16.490183 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 781/1570] 10.222.0.27 () {58 vars in 1152 bytes} [Fri Jun 10 13:03:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 13:03:16.490183 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddfd920-87af-48e4-9a41-e96ae0bc975c admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:16.500110 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddfd920-87af-48e4-9a41-e96ae0bc975c admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:16.505435 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef04a830-1bba-4bab-bae8-fd06fc910930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:16.506107 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef04a830-1bba-4bab-bae8-fd06fc910930 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:16.506393 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef04a830-1bba-4bab-bae8-fd06fc910930 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:16.506672 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef04a830-1bba-4bab-bae8-fd06fc910930 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:16.507397 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef04a830-1bba-4bab-bae8-fd06fc910930 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:16.509151 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef04a830-1bba-4bab-bae8-fd06fc910930 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:16.523090 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 782/1571] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:03:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:03:16.523090 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef04a830-1bba-4bab-bae8-fd06fc910930 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:16.531082 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef04a830-1bba-4bab-bae8-fd06fc910930 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:16.535732 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019de1a8-0779-4eea-92ae-ac4456f6b2d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:16.536408 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019de1a8-0779-4eea-92ae-ac4456f6b2d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:16.536691 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019de1a8-0779-4eea-92ae-ac4456f6b2d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:16.536955 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019de1a8-0779-4eea-92ae-ac4456f6b2d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:16.537633 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019de1a8-0779-4eea-92ae-ac4456f6b2d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:16.539543 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019de1a8-0779-4eea-92ae-ac4456f6b2d3 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:16.579092 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 790/1572] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:03:16 2022] POST /identity/v3/projects => generated 383 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:03:16.580002 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81c47c1-52d7-47a0-a923-8ebe16048341 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:16.639243 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81c47c1-52d7-47a0-a923-8ebe16048341 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:16.643961 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ca80e7-d6fc-4b4a-a4ce-40976bb9a45d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:16.648033 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ca80e7-d6fc-4b4a-a4ce-40976bb9a45d admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:16.648033 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ca80e7-d6fc-4b4a-a4ce-40976bb9a45d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:16.648033 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ca80e7-d6fc-4b4a-a4ce-40976bb9a45d admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:16.648033 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ca80e7-d6fc-4b4a-a4ce-40976bb9a45d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:16.648504 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 783/1573] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:03:16 2022] POST /identity/v3/projects => generated 393 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:03:16.648608 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ca80e7-d6fc-4b4a-a4ce-40976bb9a45d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:16.648697 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019de1a8-0779-4eea-92ae-ac4456f6b2d3 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:16.709006 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019de1a8-0779-4eea-92ae-ac4456f6b2d3 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:16.713777 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db027f3-00dd-496e-a9c6-c334e91b5c1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:16.714436 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db027f3-00dd-496e-a9c6-c334e91b5c1b admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:16.714723 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db027f3-00dd-496e-a9c6-c334e91b5c1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:16.714999 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db027f3-00dd-496e-a9c6-c334e91b5c1b admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:16.715679 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db027f3-00dd-496e-a9c6-c334e91b5c1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:16.718188 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db027f3-00dd-496e-a9c6-c334e91b5c1b admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:16.734226 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 791/1574] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:03:16 2022] POST /identity/v3/projects => generated 371 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:03:16.751186 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ca80e7-d6fc-4b4a-a4ce-40976bb9a45d admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:16.795192 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ca80e7-d6fc-4b4a-a4ce-40976bb9a45d admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:16.799716 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d4a8d7-8878-48f2-ad3a-22e43d645ceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:16.800410 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d4a8d7-8878-48f2-ad3a-22e43d645ceb admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:16.800698 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d4a8d7-8878-48f2-ad3a-22e43d645ceb admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:16.800976 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d4a8d7-8878-48f2-ad3a-22e43d645ceb admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:16.801653 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d4a8d7-8878-48f2-ad3a-22e43d645ceb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:16.803515 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d4a8d7-8878-48f2-ad3a-22e43d645ceb admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:16.820285 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 784/1575] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:03:16 2022] POST /identity/v3/projects => generated 367 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:03:16.821223 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db027f3-00dd-496e-a9c6-c334e91b5c1b admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:16.882092 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db027f3-00dd-496e-a9c6-c334e91b5c1b admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:16.886699 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8951265b-f7d9-4b01-a5e2-214d9d495788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:16.887586 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8951265b-f7d9-4b01-a5e2-214d9d495788 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:16.888505 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8951265b-f7d9-4b01-a5e2-214d9d495788 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:16.888786 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8951265b-f7d9-4b01-a5e2-214d9d495788 admin admin] PATH_INFO: `/v3/projects/3835b904d7ab4046ad01c682dd49d0d7/users/83b6da423aa447f1aad0a1498f4d8f7c/roles/43106c664e1b40de984c0c62701149c8` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:16.900314 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 792/1576] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:03:16 2022] POST /identity/v3/projects => generated 373 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:03:16.901476 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d4a8d7-8878-48f2-ad3a-22e43d645ceb admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:16.904692 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8951265b-f7d9-4b01-a5e2-214d9d495788 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3835b904d7ab4046ad01c682dd49d0d7, user_id=83b6da423aa447f1aad0a1498f4d8f7c, role_id=43106c664e1b40de984c0c62701149c8)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:16.905778 dv-cinder-master-3-smb devstack@keystone.service[80238]: WARNING py.warnings [None req-8951265b-f7d9-4b01-a5e2-214d9d495788 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 10 13:03:16.905778 dv-cinder-master-3-smb devstack@keystone.service[80238]: warnings.warn(msg) Jun 10 13:03:16.905778 dv-cinder-master-3-smb devstack@keystone.service[80238]: Jun 10 13:03:16.907694 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8951265b-f7d9-4b01-a5e2-214d9d495788 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:16.909988 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:03:16.949537 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:03:16.962814 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d4a8d7-8878-48f2-ad3a-22e43d645ceb admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:16.967516 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6ecd7e-c169-48a7-988d-752d64a44a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:16.968240 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6ecd7e-c169-48a7-988d-752d64a44a99 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:16.968528 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6ecd7e-c169-48a7-988d-752d64a44a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:16.968809 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6ecd7e-c169-48a7-988d-752d64a44a99 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:16.969530 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6ecd7e-c169-48a7-988d-752d64a44a99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:16.971521 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6ecd7e-c169-48a7-988d-752d64a44a99 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:17.001535 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 785/1577] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:03:16 2022] PUT /identity/v3/projects/3835b904d7ab4046ad01c682dd49d0d7/users/83b6da423aa447f1aad0a1498f4d8f7c/roles/43106c664e1b40de984c0c62701149c8 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:03:17.002342 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8951265b-f7d9-4b01-a5e2-214d9d495788 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:17.048194 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 793/1578] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:03:16 2022] POST /identity/v3/projects => generated 385 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:03:17.049013 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6ecd7e-c169-48a7-988d-752d64a44a99 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:17.062621 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8951265b-f7d9-4b01-a5e2-214d9d495788 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:17.113551 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6ecd7e-c169-48a7-988d-752d64a44a99 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:17.118149 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc74c163-9628-4fec-8052-5501c8e97e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:17.118819 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc74c163-9628-4fec-8052-5501c8e97e6c admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:17.119103 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc74c163-9628-4fec-8052-5501c8e97e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:17.119382 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc74c163-9628-4fec-8052-5501c8e97e6c admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:17.120104 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc74c163-9628-4fec-8052-5501c8e97e6c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:17.121046 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35a1df2-3590-438e-8e7b-f2bb9cd46ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:17.121938 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35a1df2-3590-438e-8e7b-f2bb9cd46ecd admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:17.122087 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc74c163-9628-4fec-8052-5501c8e97e6c admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:17.122855 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35a1df2-3590-438e-8e7b-f2bb9cd46ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:17.123136 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35a1df2-3590-438e-8e7b-f2bb9cd46ecd admin admin] PATH_INFO: `/v3/projects/1a45808d686e417ab6e8df91aa2735a7/users/634e6116f688494c9e11d59479f94a43/roles/43106c664e1b40de984c0c62701149c8` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:17.137953 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35a1df2-3590-438e-8e7b-f2bb9cd46ecd admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a45808d686e417ab6e8df91aa2735a7, user_id=634e6116f688494c9e11d59479f94a43, role_id=43106c664e1b40de984c0c62701149c8)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:17.141054 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35a1df2-3590-438e-8e7b-f2bb9cd46ecd admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:17.218070 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 794/1579] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:03:17 2022] POST /identity/v3/users => generated 292 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:03:17.218933 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc74c163-9628-4fec-8052-5501c8e97e6c admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:17.228384 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc74c163-9628-4fec-8052-5501c8e97e6c admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:17.230516 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 786/1580] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:03:17 2022] PUT /identity/v3/projects/1a45808d686e417ab6e8df91aa2735a7/users/634e6116f688494c9e11d59479f94a43/roles/43106c664e1b40de984c0c62701149c8 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:03:17.231365 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35a1df2-3590-438e-8e7b-f2bb9cd46ecd admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:17.289065 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40940f90-8aa3-4959-8cbd-db4ffe5a9911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:17.289735 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40940f90-8aa3-4959-8cbd-db4ffe5a9911 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:17.290015 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40940f90-8aa3-4959-8cbd-db4ffe5a9911 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:17.290291 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40940f90-8aa3-4959-8cbd-db4ffe5a9911 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:17.290967 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40940f90-8aa3-4959-8cbd-db4ffe5a9911 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:17.292572 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35a1df2-3590-438e-8e7b-f2bb9cd46ecd admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:17.293157 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40940f90-8aa3-4959-8cbd-db4ffe5a9911 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:17.297670 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8a84db-3cf1-4780-9046-bc4d59d7ffb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:17.298383 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8a84db-3cf1-4780-9046-bc4d59d7ffb0 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:17.298618 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8a84db-3cf1-4780-9046-bc4d59d7ffb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:17.298897 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8a84db-3cf1-4780-9046-bc4d59d7ffb0 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:17.299578 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8a84db-3cf1-4780-9046-bc4d59d7ffb0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:17.301517 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8a84db-3cf1-4780-9046-bc4d59d7ffb0 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:17.390935 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 795/1581] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:03:17 2022] POST /identity/v3/users => generated 297 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:03:17.390935 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40940f90-8aa3-4959-8cbd-db4ffe5a9911 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:17.398136 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 787/1582] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:03:17 2022] POST /identity/v3/users => generated 286 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:03:17.398136 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8a84db-3cf1-4780-9046-bc4d59d7ffb0 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:17.399106 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40940f90-8aa3-4959-8cbd-db4ffe5a9911 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:17.404306 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fb3210-db1d-4b24-93fa-9109139014ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:17.404957 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fb3210-db1d-4b24-93fa-9109139014ad admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:17.405239 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fb3210-db1d-4b24-93fa-9109139014ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:17.405522 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fb3210-db1d-4b24-93fa-9109139014ad admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:17.405667 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8a84db-3cf1-4780-9046-bc4d59d7ffb0 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:17.406200 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fb3210-db1d-4b24-93fa-9109139014ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:17.408170 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fb3210-db1d-4b24-93fa-9109139014ad admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:17.410178 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9774c676-19d8-4488-9d33-ca126de0e354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:17.410821 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9774c676-19d8-4488-9d33-ca126de0e354 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:17.411104 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9774c676-19d8-4488-9d33-ca126de0e354 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:17.411381 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9774c676-19d8-4488-9d33-ca126de0e354 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:17.412084 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9774c676-19d8-4488-9d33-ca126de0e354 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:17.413978 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9774c676-19d8-4488-9d33-ca126de0e354 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:17.423762 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:03:17.485504 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 796/1583] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:03:17 2022] POST /identity/v3/users => generated 284 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:03:17.486988 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7058f1b8-2634-4e6a-86d8-1cfef8dbb153 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:17.487273 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7058f1b8-2634-4e6a-86d8-1cfef8dbb153 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:17.487550 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7058f1b8-2634-4e6a-86d8-1cfef8dbb153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:17.504288 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-7058f1b8-2634-4e6a-86d8-1cfef8dbb153 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:03:17.505472 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-7058f1b8-2634-4e6a-86d8-1cfef8dbb153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:17.513693 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 788/1584] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:03:17 2022] POST /identity/v3/users => generated 287 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:03:17.513693 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9774c676-19d8-4488-9d33-ca126de0e354 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:17.514303 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 797/1585] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:03:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:03:17.514408 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7058f1b8-2634-4e6a-86d8-1cfef8dbb153 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:17.524619 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9774c676-19d8-4488-9d33-ca126de0e354 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:17.525756 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7058f1b8-2634-4e6a-86d8-1cfef8dbb153 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:17.530041 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025ce5be-eb0e-4ec5-b5ac-2c3553988663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:17.530703 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025ce5be-eb0e-4ec5-b5ac-2c3553988663 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:17.533266 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5f1f5f-b155-4f44-aa90-82ed53b28a98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:17.533401 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025ce5be-eb0e-4ec5-b5ac-2c3553988663 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:17.533401 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025ce5be-eb0e-4ec5-b5ac-2c3553988663 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:17.533589 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5f1f5f-b155-4f44-aa90-82ed53b28a98 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:17.533589 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5f1f5f-b155-4f44-aa90-82ed53b28a98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:17.533737 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025ce5be-eb0e-4ec5-b5ac-2c3553988663 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:17.533821 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5f1f5f-b155-4f44-aa90-82ed53b28a98 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:17.533821 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5f1f5f-b155-4f44-aa90-82ed53b28a98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:17.533979 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025ce5be-eb0e-4ec5-b5ac-2c3553988663 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:17.534289 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5f1f5f-b155-4f44-aa90-82ed53b28a98 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:17.545749 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 798/1586] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:03:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:03:17.546969 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d75a627-2b40-4f81-9d3d-22929c061527 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:17.547261 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d75a627-2b40-4f81-9d3d-22929c061527 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:17.547561 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d75a627-2b40-4f81-9d3d-22929c061527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:17.564223 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-5d75a627-2b40-4f81-9d3d-22929c061527 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:03:17.565351 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5d75a627-2b40-4f81-9d3d-22929c061527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:17.572896 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 799/1587] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:03:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:03:17.573639 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d75a627-2b40-4f81-9d3d-22929c061527 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:17.582951 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d75a627-2b40-4f81-9d3d-22929c061527 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:17.587549 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf6ae36-02d7-4fd8-9882-e6528f198f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:17.588519 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf6ae36-02d7-4fd8-9882-e6528f198f07 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:17.588805 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf6ae36-02d7-4fd8-9882-e6528f198f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:17.589081 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf6ae36-02d7-4fd8-9882-e6528f198f07 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:17.589799 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf6ae36-02d7-4fd8-9882-e6528f198f07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:17.591927 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf6ae36-02d7-4fd8-9882-e6528f198f07 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:17.597747 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:03:17.602277 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 789/1588] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:03:17 2022] POST /identity/v3/users => generated 293 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:03:17.602749 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025ce5be-eb0e-4ec5-b5ac-2c3553988663 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:17.603238 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 800/1589] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:03:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:03:17.604183 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf6ae36-02d7-4fd8-9882-e6528f198f07 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:17.613302 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025ce5be-eb0e-4ec5-b5ac-2c3553988663 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:17.614385 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf6ae36-02d7-4fd8-9882-e6528f198f07 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:17.617984 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0a5551-5fea-4e35-beb4-3ea44207d249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:17.618645 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0a5551-5fea-4e35-beb4-3ea44207d249 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:17.618909 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0a5551-5fea-4e35-beb4-3ea44207d249 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:17.619038 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a39f72-5b4d-4fe3-bb5e-95a695ca1699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:17.619208 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0a5551-5fea-4e35-beb4-3ea44207d249 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:17.619561 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a39f72-5b4d-4fe3-bb5e-95a695ca1699 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:17.619846 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a39f72-5b4d-4fe3-bb5e-95a695ca1699 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:17.619992 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0a5551-5fea-4e35-beb4-3ea44207d249 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:17.620161 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a39f72-5b4d-4fe3-bb5e-95a695ca1699 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:17.620876 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a39f72-5b4d-4fe3-bb5e-95a695ca1699 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:17.621526 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0a5551-5fea-4e35-beb4-3ea44207d249 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:17.622424 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a39f72-5b4d-4fe3-bb5e-95a695ca1699 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:17.632235 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 801/1590] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:03:17 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:03:17.638443 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 790/1591] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:03:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:03:17.638561 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a39f72-5b4d-4fe3-bb5e-95a695ca1699 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:17.638678 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0a5551-5fea-4e35-beb4-3ea44207d249 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:17.643345 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a39f72-5b4d-4fe3-bb5e-95a695ca1699 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:17.648061 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59db71a-87c0-4f32-bb44-a8994af9a95a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:17.648704 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59db71a-87c0-4f32-bb44-a8994af9a95a admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:17.648991 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59db71a-87c0-4f32-bb44-a8994af9a95a admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:17.649273 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59db71a-87c0-4f32-bb44-a8994af9a95a admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:17.649984 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59db71a-87c0-4f32-bb44-a8994af9a95a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:17.651942 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59db71a-87c0-4f32-bb44-a8994af9a95a admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:17.663373 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 802/1592] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:03:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:03:17.664732 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59db71a-87c0-4f32-bb44-a8994af9a95a admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:17.674626 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59db71a-87c0-4f32-bb44-a8994af9a95a admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:17.679122 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e248f8-4983-41e8-9cbf-85d468452b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:17.679762 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e248f8-4983-41e8-9cbf-85d468452b4c admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:17.680080 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e248f8-4983-41e8-9cbf-85d468452b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:17.680360 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e248f8-4983-41e8-9cbf-85d468452b4c admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:17.681069 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e248f8-4983-41e8-9cbf-85d468452b4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:17.682626 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e248f8-4983-41e8-9cbf-85d468452b4c admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:17.692809 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 803/1593] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:03:17 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:03:17.693971 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e248f8-4983-41e8-9cbf-85d468452b4c admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:17.694189 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0a5551-5fea-4e35-beb4-3ea44207d249 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:17.698544 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c22eaa-3df3-4211-a954-e8b4a6fbb71e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:17.699302 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c22eaa-3df3-4211-a954-e8b4a6fbb71e service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:17.699590 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c22eaa-3df3-4211-a954-e8b4a6fbb71e service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:17.699948 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c22eaa-3df3-4211-a954-e8b4a6fbb71e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:17.703562 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e248f8-4983-41e8-9cbf-85d468452b4c admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:17.707433 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e37bb17-780c-465d-b969-bdc9081de701 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:17.708169 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e37bb17-780c-465d-b969-bdc9081de701 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:17.708519 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e37bb17-780c-465d-b969-bdc9081de701 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:17.708800 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e37bb17-780c-465d-b969-bdc9081de701 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:17.709188 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c22eaa-3df3-4211-a954-e8b4a6fbb71e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:17.710624 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c22eaa-3df3-4211-a954-e8b4a6fbb71e service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:17.720728 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 791/1594] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:03:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:03:17.721526 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c22eaa-3df3-4211-a954-e8b4a6fbb71e service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:17.724046 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e37bb17-780c-465d-b969-bdc9081de701 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:17.725522 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e37bb17-780c-465d-b969-bdc9081de701 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:17.730682 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c22eaa-3df3-4211-a954-e8b4a6fbb71e service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:17.737720 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060be2fb-ee93-4126-9be6-fbe8a67ef690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:17.738883 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060be2fb-ee93-4126-9be6-fbe8a67ef690 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:17.738883 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060be2fb-ee93-4126-9be6-fbe8a67ef690 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:17.739103 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060be2fb-ee93-4126-9be6-fbe8a67ef690 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:17.740057 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060be2fb-ee93-4126-9be6-fbe8a67ef690 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:17.741132 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 804/1595] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:03:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:03:17.741808 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060be2fb-ee93-4126-9be6-fbe8a67ef690 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:17.742159 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e37bb17-780c-465d-b969-bdc9081de701 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:17.751618 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e37bb17-780c-465d-b969-bdc9081de701 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:17.753509 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 792/1596] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:03:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:03:17.754234 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060be2fb-ee93-4126-9be6-fbe8a67ef690 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:17.759719 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acd1d8b-98cd-4fbc-80a8-90e525b89ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:17.760498 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acd1d8b-98cd-4fbc-80a8-90e525b89ef8 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:17.760837 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acd1d8b-98cd-4fbc-80a8-90e525b89ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:17.761348 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acd1d8b-98cd-4fbc-80a8-90e525b89ef8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:17.761969 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acd1d8b-98cd-4fbc-80a8-90e525b89ef8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:17.763741 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acd1d8b-98cd-4fbc-80a8-90e525b89ef8 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:17.765131 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060be2fb-ee93-4126-9be6-fbe8a67ef690 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:17.770107 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3a067d-11ec-4809-9a91-c1322a9bf078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:17.770768 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3a067d-11ec-4809-9a91-c1322a9bf078 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:17.771058 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3a067d-11ec-4809-9a91-c1322a9bf078 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:17.771337 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3a067d-11ec-4809-9a91-c1322a9bf078 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:17.772093 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3a067d-11ec-4809-9a91-c1322a9bf078 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:17.773687 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3a067d-11ec-4809-9a91-c1322a9bf078 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:17.774700 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 805/1597] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:03:17 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:03:17.775901 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acd1d8b-98cd-4fbc-80a8-90e525b89ef8 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:17.786121 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acd1d8b-98cd-4fbc-80a8-90e525b89ef8 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:17.790616 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 793/1598] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:03:17 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:03:17.790616 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3a067d-11ec-4809-9a91-c1322a9bf078 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:17.797291 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191dce19-84c4-4795-8f45-8fba7d052ada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:17.797291 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191dce19-84c4-4795-8f45-8fba7d052ada admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:17.797291 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191dce19-84c4-4795-8f45-8fba7d052ada admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:17.797291 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191dce19-84c4-4795-8f45-8fba7d052ada admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:17.797709 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191dce19-84c4-4795-8f45-8fba7d052ada admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:17.797868 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191dce19-84c4-4795-8f45-8fba7d052ada admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:17.800625 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3a067d-11ec-4809-9a91-c1322a9bf078 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:17.805322 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501fa7cb-8799-43a9-8fdc-2601bfca9085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:17.805970 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501fa7cb-8799-43a9-8fdc-2601bfca9085 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:17.806259 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501fa7cb-8799-43a9-8fdc-2601bfca9085 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:17.806536 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501fa7cb-8799-43a9-8fdc-2601bfca9085 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:17.807283 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501fa7cb-8799-43a9-8fdc-2601bfca9085 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:17.807785 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 806/1599] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:03:17 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:03:17.808651 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191dce19-84c4-4795-8f45-8fba7d052ada admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:17.808905 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501fa7cb-8799-43a9-8fdc-2601bfca9085 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:17.831158 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 794/1600] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:03:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:03:17.831345 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191dce19-84c4-4795-8f45-8fba7d052ada admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:17.831438 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501fa7cb-8799-43a9-8fdc-2601bfca9085 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:17.831529 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30984326-4225-457c-869b-16c276bfb045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:17.831529 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30984326-4225-457c-869b-16c276bfb045 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:17.831529 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30984326-4225-457c-869b-16c276bfb045 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:17.831529 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30984326-4225-457c-869b-16c276bfb045 admin admin] PATH_INFO: `/v3/projects/d06694ad116841d49f4bb980042ca4c3/users/8e32dbeb85bd4fb889d6dde932c60853/roles/43106c664e1b40de984c0c62701149c8` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:17.831842 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501fa7cb-8799-43a9-8fdc-2601bfca9085 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:17.837193 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440e0bbd-f68f-4a2e-bb4c-e22391db23c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:17.837896 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440e0bbd-f68f-4a2e-bb4c-e22391db23c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:17.838191 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440e0bbd-f68f-4a2e-bb4c-e22391db23c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:17.838506 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440e0bbd-f68f-4a2e-bb4c-e22391db23c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:17.839379 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440e0bbd-f68f-4a2e-bb4c-e22391db23c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:17.841720 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440e0bbd-f68f-4a2e-bb4c-e22391db23c7 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:17.842214 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30984326-4225-457c-869b-16c276bfb045 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d06694ad116841d49f4bb980042ca4c3, user_id=8e32dbeb85bd4fb889d6dde932c60853, role_id=43106c664e1b40de984c0c62701149c8)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:17.845306 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30984326-4225-457c-869b-16c276bfb045 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:17.853605 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 795/1601] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:03:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:03:17.853605 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440e0bbd-f68f-4a2e-bb4c-e22391db23c7 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:17.867019 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440e0bbd-f68f-4a2e-bb4c-e22391db23c7 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:17.871648 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8ff351-6f09-47a1-8ddb-074cf3c92c7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:17.872680 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8ff351-6f09-47a1-8ddb-074cf3c92c7d admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:17.872964 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8ff351-6f09-47a1-8ddb-074cf3c92c7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:17.873246 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8ff351-6f09-47a1-8ddb-074cf3c92c7d admin admin] PATH_INFO: `/v3/projects/bad59dfe0332406d973508224a0162e4/users/e0b63da5bf0b4410acc9ff573064f8ec/roles/43106c664e1b40de984c0c62701149c8` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:17.888546 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8ff351-6f09-47a1-8ddb-074cf3c92c7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=bad59dfe0332406d973508224a0162e4, user_id=e0b63da5bf0b4410acc9ff573064f8ec, role_id=43106c664e1b40de984c0c62701149c8)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:17.891217 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8ff351-6f09-47a1-8ddb-074cf3c92c7d admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:17.931540 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 807/1602] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:03:17 2022] PUT /identity/v3/projects/d06694ad116841d49f4bb980042ca4c3/users/8e32dbeb85bd4fb889d6dde932c60853/roles/43106c664e1b40de984c0c62701149c8 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:03:17.932775 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30984326-4225-457c-869b-16c276bfb045 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:17.968186 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 796/1603] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:03:17 2022] PUT /identity/v3/projects/bad59dfe0332406d973508224a0162e4/users/e0b63da5bf0b4410acc9ff573064f8ec/roles/43106c664e1b40de984c0c62701149c8 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:03:17.969009 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8ff351-6f09-47a1-8ddb-074cf3c92c7d admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:18.005518 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30984326-4225-457c-869b-16c276bfb045 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:18.040903 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8ff351-6f09-47a1-8ddb-074cf3c92c7d admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:18.045630 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738582dc-9821-416a-a27e-40e3f97f9fca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:18.046526 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738582dc-9821-416a-a27e-40e3f97f9fca admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:18.046813 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738582dc-9821-416a-a27e-40e3f97f9fca admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:18.047094 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738582dc-9821-416a-a27e-40e3f97f9fca admin admin] PATH_INFO: `/v3/projects/2befb4791c1143218332a2419ec51324/users/782a5e6223c84471b2442ee0ff0eacf1/roles/43106c664e1b40de984c0c62701149c8` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:18.050817 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:03:18.061814 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738582dc-9821-416a-a27e-40e3f97f9fca admin admin] RBAC: Authorizing `identity:create_grant(project_id=2befb4791c1143218332a2419ec51324, user_id=782a5e6223c84471b2442ee0ff0eacf1, role_id=43106c664e1b40de984c0c62701149c8)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:18.064556 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738582dc-9821-416a-a27e-40e3f97f9fca admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:18.077122 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90156e26-0ea3-4f94-806d-4e928041f588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:18.078062 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90156e26-0ea3-4f94-806d-4e928041f588 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:18.078349 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90156e26-0ea3-4f94-806d-4e928041f588 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:18.082472 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90156e26-0ea3-4f94-806d-4e928041f588 admin admin] PATH_INFO: `/v3/projects/7334d4dfa33941458c237db1a6c32809/users/655b6e90e39c46718573d84d54e77219/roles/43106c664e1b40de984c0c62701149c8` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:18.094900 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90156e26-0ea3-4f94-806d-4e928041f588 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7334d4dfa33941458c237db1a6c32809, user_id=655b6e90e39c46718573d84d54e77219, role_id=43106c664e1b40de984c0c62701149c8)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:18.098170 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90156e26-0ea3-4f94-806d-4e928041f588 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:18.180332 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 797/1604] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:03:17 2022] PUT /identity/v3/projects/2befb4791c1143218332a2419ec51324/users/782a5e6223c84471b2442ee0ff0eacf1/roles/43106c664e1b40de984c0c62701149c8 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:03:18.181463 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738582dc-9821-416a-a27e-40e3f97f9fca admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:18.230704 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:03:18.243774 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 808/1605] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:03:17 2022] PUT /identity/v3/projects/7334d4dfa33941458c237db1a6c32809/users/655b6e90e39c46718573d84d54e77219/roles/43106c664e1b40de984c0c62701149c8 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:03:18.252386 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90156e26-0ea3-4f94-806d-4e928041f588 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:18.255205 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738582dc-9821-416a-a27e-40e3f97f9fca admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:18.315433 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af9b17d-fbc6-4566-929f-dcf9d6290086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:18.316415 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af9b17d-fbc6-4566-929f-dcf9d6290086 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:18.316743 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af9b17d-fbc6-4566-929f-dcf9d6290086 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:18.317167 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af9b17d-fbc6-4566-929f-dcf9d6290086 admin admin] PATH_INFO: `/v3/projects/6cfc84b3dd644da3b65856da2defb668/users/65c8ae2f1d1b4eee8303e79a3d51769f/roles/43106c664e1b40de984c0c62701149c8` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:18.317966 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90156e26-0ea3-4f94-806d-4e928041f588 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:18.324372 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd65b40-8d59-4c6a-921b-df9ce70b3d5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:18.326659 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd65b40-8d59-4c6a-921b-df9ce70b3d5d admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:18.326659 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd65b40-8d59-4c6a-921b-df9ce70b3d5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:18.326659 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd65b40-8d59-4c6a-921b-df9ce70b3d5d admin admin] PATH_INFO: `/v3/projects/2ffa0e3d9fb04472ab9543a7737b87ae/users/c71b903e00214894b94b03b3401607a0/roles/43106c664e1b40de984c0c62701149c8` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:18.334721 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af9b17d-fbc6-4566-929f-dcf9d6290086 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cfc84b3dd644da3b65856da2defb668, user_id=65c8ae2f1d1b4eee8303e79a3d51769f, role_id=43106c664e1b40de984c0c62701149c8)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:18.337339 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af9b17d-fbc6-4566-929f-dcf9d6290086 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:18.341141 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd65b40-8d59-4c6a-921b-df9ce70b3d5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ffa0e3d9fb04472ab9543a7737b87ae, user_id=c71b903e00214894b94b03b3401607a0, role_id=43106c664e1b40de984c0c62701149c8)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:18.344762 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd65b40-8d59-4c6a-921b-df9ce70b3d5d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:18.411694 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 798/1606] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:03:18 2022] PUT /identity/v3/projects/6cfc84b3dd644da3b65856da2defb668/users/65c8ae2f1d1b4eee8303e79a3d51769f/roles/43106c664e1b40de984c0c62701149c8 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:03:18.417483 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d61ca4-6c2a-4178-9b48-cc1781f7520a None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:18.417483 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d61ca4-6c2a-4178-9b48-cc1781f7520a None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:18.417483 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d61ca4-6c2a-4178-9b48-cc1781f7520a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:18.437243 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-76d61ca4-6c2a-4178-9b48-cc1781f7520a None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:03:18.443502 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-76d61ca4-6c2a-4178-9b48-cc1781f7520a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:18.469741 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 809/1607] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:03:18 2022] PUT /identity/v3/projects/2ffa0e3d9fb04472ab9543a7737b87ae/users/c71b903e00214894b94b03b3401607a0/roles/43106c664e1b40de984c0c62701149c8 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:03:18.470792 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ecea80-054f-4a59-8ba6-53db632c7a25 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:18.471081 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ecea80-054f-4a59-8ba6-53db632c7a25 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:18.471359 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ecea80-054f-4a59-8ba6-53db632c7a25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:18.502838 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-46ecea80-054f-4a59-8ba6-53db632c7a25 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:03:18.505163 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-46ecea80-054f-4a59-8ba6-53db632c7a25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:18.521961 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 799/1608] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:03:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:03:18.524880 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d36b07-40b3-46b3-b749-1805c4f2fc8a None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:18.525745 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d36b07-40b3-46b3-b749-1805c4f2fc8a None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:18.527589 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d36b07-40b3-46b3-b749-1805c4f2fc8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:18.587532 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-19d36b07-40b3-46b3-b749-1805c4f2fc8a None None] MFA Rules not processed for user `782a5e6223c84471b2442ee0ff0eacf1`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:03:18.588903 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-19d36b07-40b3-46b3-b749-1805c4f2fc8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:18.593915 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 810/1609] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:03:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:03:18.593915 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eee4f89-37b4-47a5-a872-d78d8f73f66f None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:18.593915 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eee4f89-37b4-47a5-a872-d78d8f73f66f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:18.593915 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eee4f89-37b4-47a5-a872-d78d8f73f66f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:18.594673 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:03:18.629004 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-9eee4f89-37b4-47a5-a872-d78d8f73f66f None None] MFA Rules not processed for user `655b6e90e39c46718573d84d54e77219`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:03:18.630563 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9eee4f89-37b4-47a5-a872-d78d8f73f66f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:18.674416 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 800/1610] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:03:18 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:03:18.674416 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6756de-bc5a-430e-a7dd-f7d09d29d2bd None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:18.674416 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6756de-bc5a-430e-a7dd-f7d09d29d2bd None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:18.674416 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6756de-bc5a-430e-a7dd-f7d09d29d2bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:18.711222 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-7c6756de-bc5a-430e-a7dd-f7d09d29d2bd None None] MFA Rules not processed for user `65c8ae2f1d1b4eee8303e79a3d51769f`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:03:18.712936 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-7c6756de-bc5a-430e-a7dd-f7d09d29d2bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:18.730223 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 811/1611] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:03:18 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:03:18.730223 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f88937f-358e-4458-8a0f-ca6d9b70f105 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:18.730223 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f88937f-358e-4458-8a0f-ca6d9b70f105 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:18.730223 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f88937f-358e-4458-8a0f-ca6d9b70f105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:18.755378 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-2f88937f-358e-4458-8a0f-ca6d9b70f105 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:03:18.757635 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2f88937f-358e-4458-8a0f-ca6d9b70f105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:18.772246 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 812/1612] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:03:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:03:18.773839 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f88937f-358e-4458-8a0f-ca6d9b70f105 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:18.777044 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2f88937f-358e-4458-8a0f-ca6d9b70f105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:18.791242 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:03:18.801539 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 801/1613] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:03:18 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:03:18.802579 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6756de-bc5a-430e-a7dd-f7d09d29d2bd None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:18.804388 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-7c6756de-bc5a-430e-a7dd-f7d09d29d2bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:18.870445 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f88937f-358e-4458-8a0f-ca6d9b70f105 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:18.877013 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6756de-bc5a-430e-a7dd-f7d09d29d2bd None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:18.880455 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2605212a-f683-4b40-aaac-3fc5de674b9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:18.880906 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7d143c-83d8-4113-9d94-9257de7417ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:18.881137 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2605212a-f683-4b40-aaac-3fc5de674b9b service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:18.881421 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2605212a-f683-4b40-aaac-3fc5de674b9b service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:18.881593 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d143c-83d8-4113-9d94-9257de7417ff service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:18.881702 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2605212a-f683-4b40-aaac-3fc5de674b9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:18.881877 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d143c-83d8-4113-9d94-9257de7417ff service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:18.882108 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7d143c-83d8-4113-9d94-9257de7417ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:18.891799 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7d143c-83d8-4113-9d94-9257de7417ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:18.894079 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7d143c-83d8-4113-9d94-9257de7417ff service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:18.896157 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2605212a-f683-4b40-aaac-3fc5de674b9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:18.897652 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2605212a-f683-4b40-aaac-3fc5de674b9b service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:18.907069 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 802/1614] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:03:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:03:18.908794 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7d143c-83d8-4113-9d94-9257de7417ff service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:18.909507 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 813/1615] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:03:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:03:18.911201 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b303f015-21a6-4744-9eae-88234d4becb3 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:18.911589 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b303f015-21a6-4744-9eae-88234d4becb3 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:18.911957 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b303f015-21a6-4744-9eae-88234d4becb3 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:18.922568 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 814/1616] 10.222.0.27 () {54 vars in 961 bytes} [Fri Jun 10 13:03:18 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 13:03:18.922568 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b303f015-21a6-4744-9eae-88234d4becb3 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:18.934069 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b303f015-21a6-4744-9eae-88234d4becb3 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:18.940251 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a132d43b-1e5a-49a2-9322-f17c1bbe3fc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:18.941012 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a132d43b-1e5a-49a2-9322-f17c1bbe3fc9 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:18.941360 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a132d43b-1e5a-49a2-9322-f17c1bbe3fc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:18.941733 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a132d43b-1e5a-49a2-9322-f17c1bbe3fc9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:18.942571 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a132d43b-1e5a-49a2-9322-f17c1bbe3fc9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:18.945035 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a132d43b-1e5a-49a2-9322-f17c1bbe3fc9 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:18.957220 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:03:18.971570 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7d143c-83d8-4113-9d94-9257de7417ff service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:18.975370 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7e69cd-28dc-41c5-8cf2-1a8dab3773af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6cc95a563b941ac92ab283aa0d5a5d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:18.978106 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7e69cd-28dc-41c5-8cf2-1a8dab3773af service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:18.978106 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7e69cd-28dc-41c5-8cf2-1a8dab3773af service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:18.978106 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7e69cd-28dc-41c5-8cf2-1a8dab3773af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:18.985491 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7e69cd-28dc-41c5-8cf2-1a8dab3773af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:18.986908 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7e69cd-28dc-41c5-8cf2-1a8dab3773af service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:18.995599 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 803/1617] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:03:18 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:03:18.996373 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7e69cd-28dc-41c5-8cf2-1a8dab3773af service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:19.005341 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7e69cd-28dc-41c5-8cf2-1a8dab3773af service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:19.009153 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d20dc63-cca9-49ca-b69d-87e9e52b042b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:19.009802 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d20dc63-cca9-49ca-b69d-87e9e52b042b service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:19.010087 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d20dc63-cca9-49ca-b69d-87e9e52b042b service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:19.010372 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d20dc63-cca9-49ca-b69d-87e9e52b042b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:19.021623 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d20dc63-cca9-49ca-b69d-87e9e52b042b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:19.023379 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d20dc63-cca9-49ca-b69d-87e9e52b042b service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:19.023934 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 815/1618] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:03:18 2022] POST /identity/v3/projects => generated 373 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:03:19.025769 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1bb707-bde0-486e-82b2-1290c43a629b None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:19.025998 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1bb707-bde0-486e-82b2-1290c43a629b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:19.026193 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1bb707-bde0-486e-82b2-1290c43a629b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:19.036122 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 804/1619] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:03:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:03:19.036762 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d20dc63-cca9-49ca-b69d-87e9e52b042b service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:19.051189 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d20dc63-cca9-49ca-b69d-87e9e52b042b service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:19.059301 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-2f1bb707-bde0-486e-82b2-1290c43a629b None None] MFA Rules not processed for user `8bd6caa461e84100a3d5fe5af5d80f3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:03:19.061987 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2f1bb707-bde0-486e-82b2-1290c43a629b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:19.136366 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538f1f09-c67c-4a29-83ca-0e15edf2031d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:19.138324 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538f1f09-c67c-4a29-83ca-0e15edf2031d admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:19.138677 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538f1f09-c67c-4a29-83ca-0e15edf2031d admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:19.138960 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538f1f09-c67c-4a29-83ca-0e15edf2031d admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:19.140803 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538f1f09-c67c-4a29-83ca-0e15edf2031d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:19.143317 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538f1f09-c67c-4a29-83ca-0e15edf2031d admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:19.183797 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 816/1620] 10.222.0.27 () {58 vars in 1058 bytes} [Fri Jun 10 13:03:19 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:03:19.192474 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1bb707-bde0-486e-82b2-1290c43a629b None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:19.206508 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1bb707-bde0-486e-82b2-1290c43a629b None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:19.215559 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5d563f-b96a-46b0-8b3a-6123840a5a3e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bd6caa461e84100a3d5fe5af5d80f3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:19.217827 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5d563f-b96a-46b0-8b3a-6123840a5a3e service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:19.217827 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5d563f-b96a-46b0-8b3a-6123840a5a3e service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:19.217827 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5d563f-b96a-46b0-8b3a-6123840a5a3e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:19.270051 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 805/1621] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:03:19 2022] POST /identity/v3/users => generated 293 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:03:19.274027 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538f1f09-c67c-4a29-83ca-0e15edf2031d admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:19.296720 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538f1f09-c67c-4a29-83ca-0e15edf2031d admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:19.310797 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e29799-2bfa-44c7-9ad6-df47d6c422ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:19.311511 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e29799-2bfa-44c7-9ad6-df47d6c422ae admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:19.312329 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e29799-2bfa-44c7-9ad6-df47d6c422ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:19.312726 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e29799-2bfa-44c7-9ad6-df47d6c422ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:19.313639 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e29799-2bfa-44c7-9ad6-df47d6c422ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:19.315265 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e29799-2bfa-44c7-9ad6-df47d6c422ae admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:19.326840 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5d563f-b96a-46b0-8b3a-6123840a5a3e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:19.329028 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5d563f-b96a-46b0-8b3a-6123840a5a3e service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:19.338355 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 806/1622] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:03:19 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:03:19.339255 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e29799-2bfa-44c7-9ad6-df47d6c422ae admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:19.359320 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 817/1623] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:03:19 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:03:19.362150 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fed6ca2-1a56-4388-8219-f05711efd9ab None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:19.362447 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fed6ca2-1a56-4388-8219-f05711efd9ab None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:19.362734 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fed6ca2-1a56-4388-8219-f05711efd9ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:19.385769 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e29799-2bfa-44c7-9ad6-df47d6c422ae admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:19.404609 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-7fed6ca2-1a56-4388-8219-f05711efd9ab None None] MFA Rules not processed for user `83b6da423aa447f1aad0a1498f4d8f7c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:03:19.405993 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-7fed6ca2-1a56-4388-8219-f05711efd9ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:19.412874 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d48c885-f6e4-4d6c-96f0-38297acc2bc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:19.413788 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d48c885-f6e4-4d6c-96f0-38297acc2bc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:19.414074 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d48c885-f6e4-4d6c-96f0-38297acc2bc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:19.414349 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d48c885-f6e4-4d6c-96f0-38297acc2bc0 admin admin] PATH_INFO: `/v3/projects/03b2274321df4dbcaa54ce9ab1cbce64/users/979f3e87ea42484d93c6340dba790e83/roles/b43d5a8247e244869c2c70f986f42e8a` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:19.457551 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d48c885-f6e4-4d6c-96f0-38297acc2bc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03b2274321df4dbcaa54ce9ab1cbce64, user_id=979f3e87ea42484d93c6340dba790e83, role_id=b43d5a8247e244869c2c70f986f42e8a)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:19.459615 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:03:19.462281 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d48c885-f6e4-4d6c-96f0-38297acc2bc0 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:19.504840 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 818/1624] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:03:19 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:03:19.531768 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4c733b-8416-4b43-afe3-49db808a6582 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:19.532201 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4c733b-8416-4b43-afe3-49db808a6582 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:19.532490 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4c733b-8416-4b43-afe3-49db808a6582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:19.550476 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 807/1625] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:03:19 2022] PUT /identity/v3/projects/03b2274321df4dbcaa54ce9ab1cbce64/users/979f3e87ea42484d93c6340dba790e83/roles/b43d5a8247e244869c2c70f986f42e8a => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:03:19.554190 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d48c885-f6e4-4d6c-96f0-38297acc2bc0 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:19.594125 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-9b4c733b-8416-4b43-afe3-49db808a6582 None None] MFA Rules not processed for user `83b6da423aa447f1aad0a1498f4d8f7c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:03:19.595352 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9b4c733b-8416-4b43-afe3-49db808a6582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:19.673823 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d48c885-f6e4-4d6c-96f0-38297acc2bc0 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:19.679265 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703f5feb-6da7-4c23-8148-055dde2762da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:19.681647 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703f5feb-6da7-4c23-8148-055dde2762da admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:19.681647 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703f5feb-6da7-4c23-8148-055dde2762da admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:19.681647 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703f5feb-6da7-4c23-8148-055dde2762da admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:19.681647 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703f5feb-6da7-4c23-8148-055dde2762da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:19.703677 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703f5feb-6da7-4c23-8148-055dde2762da admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:19.722872 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 819/1626] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:03:19 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:03:19.722872 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dd2c44-a7b3-41fe-a075-4fd97cb73be9 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:19.722872 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dd2c44-a7b3-41fe-a075-4fd97cb73be9 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:19.722872 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dd2c44-a7b3-41fe-a075-4fd97cb73be9 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:19.734321 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 820/1627] 10.222.0.27 () {54 vars in 912 bytes} [Fri Jun 10 13:03:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 13:03:19.738447 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 808/1628] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:03:19 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:03:19.742470 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703f5feb-6da7-4c23-8148-055dde2762da admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:19.742633 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c9776e-047d-44f6-92b8-774327d6be19 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:19.742910 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c9776e-047d-44f6-92b8-774327d6be19 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:19.743159 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c9776e-047d-44f6-92b8-774327d6be19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:19.767989 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703f5feb-6da7-4c23-8148-055dde2762da admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:19.782055 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ea6dd9-b83d-49e1-8aec-194078dc0122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:19.782928 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-c4c9776e-047d-44f6-92b8-774327d6be19 None None] MFA Rules not processed for user `8bd6caa461e84100a3d5fe5af5d80f3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:03:19.783053 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ea6dd9-b83d-49e1-8aec-194078dc0122 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:19.783410 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ea6dd9-b83d-49e1-8aec-194078dc0122 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:19.783756 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ea6dd9-b83d-49e1-8aec-194078dc0122 admin admin] PATH_INFO: `/v3/projects/03b2274321df4dbcaa54ce9ab1cbce64/users/979f3e87ea42484d93c6340dba790e83/roles/b43d5a8247e244869c2c70f986f42e8a` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:19.784201 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c4c9776e-047d-44f6-92b8-774327d6be19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:19.792553 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ea6dd9-b83d-49e1-8aec-194078dc0122 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03b2274321df4dbcaa54ce9ab1cbce64, user_id=979f3e87ea42484d93c6340dba790e83, role_id=b43d5a8247e244869c2c70f986f42e8a)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:19.802341 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ea6dd9-b83d-49e1-8aec-194078dc0122 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:19.822192 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 809/1629] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:03:19 2022] PUT /identity/v3/projects/03b2274321df4dbcaa54ce9ab1cbce64/users/979f3e87ea42484d93c6340dba790e83/roles/b43d5a8247e244869c2c70f986f42e8a => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:03:19.824805 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e36fd1b-343d-4ccd-ac33-759b1fa0269e None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:19.825343 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e36fd1b-343d-4ccd-ac33-759b1fa0269e None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:19.825920 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e36fd1b-343d-4ccd-ac33-759b1fa0269e None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:19.827315 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 810/1630] 10.222.0.27 () {54 vars in 961 bytes} [Fri Jun 10 13:03:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 13:03:19.834216 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f84c62-f8c7-49f1-b11d-1029fe0a0641 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:19.834548 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f84c62-f8c7-49f1-b11d-1029fe0a0641 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:19.834830 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f84c62-f8c7-49f1-b11d-1029fe0a0641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:19.871998 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:03:19.912239 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-c9f84c62-f8c7-49f1-b11d-1029fe0a0641 None None] MFA Rules not processed for user `979f3e87ea42484d93c6340dba790e83`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:03:19.914542 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c9f84c62-f8c7-49f1-b11d-1029fe0a0641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:19.960086 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 821/1631] 10.222.0.27 () {58 vars in 1009 bytes} [Fri Jun 10 13:03:19 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:03:19.962714 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012264e1-625f-41c6-9840-f91c1330122e None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:19.963016 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012264e1-625f-41c6-9840-f91c1330122e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:19.963305 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012264e1-625f-41c6-9840-f91c1330122e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:20.008505 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-012264e1-625f-41c6-9840-f91c1330122e None None] MFA Rules not processed for user `8bd6caa461e84100a3d5fe5af5d80f3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:03:20.012313 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-012264e1-625f-41c6-9840-f91c1330122e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:20.041821 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 811/1632] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:03:19 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:03:20.044561 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f84c62-f8c7-49f1-b11d-1029fe0a0641 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:20.122846 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f84c62-f8c7-49f1-b11d-1029fe0a0641 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:20.126537 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e73cc2-ce8e-44b1-ad33-fe5326509dd6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6cc95a563b941ac92ab283aa0d5a5d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:20.127191 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e73cc2-ce8e-44b1-ad33-fe5326509dd6 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:20.127475 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e73cc2-ce8e-44b1-ad33-fe5326509dd6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:20.127769 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e73cc2-ce8e-44b1-ad33-fe5326509dd6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:20.130791 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 822/1633] 10.222.0.27 () {58 vars in 1058 bytes} [Fri Jun 10 13:03:19 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:03:20.140018 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e73cc2-ce8e-44b1-ad33-fe5326509dd6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:20.146378 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012264e1-625f-41c6-9840-f91c1330122e None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:20.146509 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e73cc2-ce8e-44b1-ad33-fe5326509dd6 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:20.151808 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012264e1-625f-41c6-9840-f91c1330122e None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:20.154411 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 812/1634] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:03:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:03:20.158524 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d883406-6291-4a4e-8e65-059e89b16114 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bd6caa461e84100a3d5fe5af5d80f3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:20.159181 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d883406-6291-4a4e-8e65-059e89b16114 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:20.159467 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d883406-6291-4a4e-8e65-059e89b16114 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:20.159797 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d883406-6291-4a4e-8e65-059e89b16114 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:20.236079 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d883406-6291-4a4e-8e65-059e89b16114 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:20.238691 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d883406-6291-4a4e-8e65-059e89b16114 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:20.251232 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 823/1635] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:03:20 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 10 13:03:20.394978 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e73cc2-ce8e-44b1-ad33-fe5326509dd6 service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:20.406941 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e73cc2-ce8e-44b1-ad33-fe5326509dd6 service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:20.416212 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7755e0cf-4d24-4479-9386-106c457dc723 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6cc95a563b941ac92ab283aa0d5a5d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:20.416611 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7755e0cf-4d24-4479-9386-106c457dc723 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:20.416912 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7755e0cf-4d24-4479-9386-106c457dc723 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:20.417248 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7755e0cf-4d24-4479-9386-106c457dc723 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:20.432470 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7755e0cf-4d24-4479-9386-106c457dc723 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:20.433974 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7755e0cf-4d24-4479-9386-106c457dc723 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:20.448738 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 813/1636] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:03:20 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:03:20.504389 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:03:20.643587 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ba2b33-27cf-4a94-bac8-3db858c74175 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:20.645128 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ba2b33-27cf-4a94-bac8-3db858c74175 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:20.645735 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ba2b33-27cf-4a94-bac8-3db858c74175 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:20.647898 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 824/1637] 10.222.0.27 () {54 vars in 912 bytes} [Fri Jun 10 13:03:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 13:03:20.653955 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadb5e2c-9016-48db-9c09-570f1cd1cd50 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:20.655054 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadb5e2c-9016-48db-9c09-570f1cd1cd50 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:20.655611 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadb5e2c-9016-48db-9c09-570f1cd1cd50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:20.680286 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-aadb5e2c-9016-48db-9c09-570f1cd1cd50 None None] MFA Rules not processed for user `8bd6caa461e84100a3d5fe5af5d80f3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:03:20.682087 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-aadb5e2c-9016-48db-9c09-570f1cd1cd50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:20.694765 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 814/1638] 10.222.0.27 () {58 vars in 1009 bytes} [Fri Jun 10 13:03:20 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:03:20.708284 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ba2b33-27cf-4a94-bac8-3db858c74175 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:20.711470 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-77ba2b33-27cf-4a94-bac8-3db858c74175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:20.811069 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ba2b33-27cf-4a94-bac8-3db858c74175 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:20.816470 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53d472e-d4ee-427d-9764-c4323fc304dc service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d3eb01a98274050b930616b98998c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:20.817585 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d472e-d4ee-427d-9764-c4323fc304dc service glance] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:20.818112 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d472e-d4ee-427d-9764-c4323fc304dc service glance] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:20.818643 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d472e-d4ee-427d-9764-c4323fc304dc service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:20.833580 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53d472e-d4ee-427d-9764-c4323fc304dc service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:20.835717 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53d472e-d4ee-427d-9764-c4323fc304dc service glance] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:20.874176 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 825/1639] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:03:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:03:21.451168 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead07d53-6769-40f7-9eaf-5662ec7b408c None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:21.451653 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead07d53-6769-40f7-9eaf-5662ec7b408c None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:21.451917 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead07d53-6769-40f7-9eaf-5662ec7b408c None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:21.453316 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 815/1640] 10.222.0.27 () {54 vars in 912 bytes} [Fri Jun 10 13:03:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 13:03:21.473058 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfa1d23-dff7-4d65-a540-c78d3b25e864 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:21.473403 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfa1d23-dff7-4d65-a540-c78d3b25e864 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:21.473636 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfa1d23-dff7-4d65-a540-c78d3b25e864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:21.515840 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-7dfa1d23-dff7-4d65-a540-c78d3b25e864 None None] MFA Rules not processed for user `bc4415461a974c24ad7105a97c7d915b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:03:21.521840 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-7dfa1d23-dff7-4d65-a540-c78d3b25e864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:21.673139 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 826/1641] 10.222.0.27 () {58 vars in 1009 bytes} [Fri Jun 10 13:03:21 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:03:21.695343 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead07d53-6769-40f7-9eaf-5662ec7b408c None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:21.720902 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead07d53-6769-40f7-9eaf-5662ec7b408c None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:21.724922 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa134161-f46a-4c39-8812-b0cfe457e8eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:21.726279 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa134161-f46a-4c39-8812-b0cfe457e8eb service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:21.726876 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa134161-f46a-4c39-8812-b0cfe457e8eb service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:21.727419 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa134161-f46a-4c39-8812-b0cfe457e8eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:21.737596 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa134161-f46a-4c39-8812-b0cfe457e8eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:21.739836 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa134161-f46a-4c39-8812-b0cfe457e8eb service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:21.752318 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 816/1642] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:03:21 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 10 13:03:22.327823 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfa1d23-dff7-4d65-a540-c78d3b25e864 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:22.327823 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:03:22.340592 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfa1d23-dff7-4d65-a540-c78d3b25e864 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:22.341652 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdb2102-0ba9-4aa5-8c38-073b837da369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:22.342429 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdb2102-0ba9-4aa5-8c38-073b837da369 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:22.342429 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdb2102-0ba9-4aa5-8c38-073b837da369 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:22.342429 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdb2102-0ba9-4aa5-8c38-073b837da369 admin admin] PATH_INFO: `/v3/users/65c8ae2f1d1b4eee8303e79a3d51769f` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:22.348410 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdb2102-0ba9-4aa5-8c38-073b837da369 admin admin] RBAC: Authorizing `identity:delete_user(user_id=65c8ae2f1d1b4eee8303e79a3d51769f)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:22.348410 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdb2102-0ba9-4aa5-8c38-073b837da369 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:22.605915 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4cdb2102-0ba9-4aa5-8c38-073b837da369 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65c8ae2f1d1b4eee8303e79a3d51769f'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:03:22.633971 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4cdb2102-0ba9-4aa5-8c38-073b837da369 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65c8ae2f1d1b4eee8303e79a3d51769f'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:03:22.768267 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4cdb2102-0ba9-4aa5-8c38-073b837da369 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65c8ae2f1d1b4eee8303e79a3d51769f'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:03:22.769527 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4cdb2102-0ba9-4aa5-8c38-073b837da369 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65c8ae2f1d1b4eee8303e79a3d51769f'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:03:22.790120 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4cdb2102-0ba9-4aa5-8c38-073b837da369 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65c8ae2f1d1b4eee8303e79a3d51769f'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:03:22.816002 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 827/1643] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:03:22 2022] DELETE /identity/v3/users/65c8ae2f1d1b4eee8303e79a3d51769f => generated 0 bytes in 634 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 10 13:03:22.817587 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65509804-592f-45db-9c01-4f9672da4e3b None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:22.818478 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65509804-592f-45db-9c01-4f9672da4e3b None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:22.818892 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65509804-592f-45db-9c01-4f9672da4e3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:22.836500 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a392bdb1-5a77-434f-835d-6e9d2fbc53f8 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:22.836793 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a392bdb1-5a77-434f-835d-6e9d2fbc53f8 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:22.837071 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a392bdb1-5a77-434f-835d-6e9d2fbc53f8 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:22.846452 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 828/1644] 10.222.0.27 () {54 vars in 960 bytes} [Fri Jun 10 13:03:22 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 13:03:22.857467 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-65509804-592f-45db-9c01-4f9672da4e3b None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:03:22.863168 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-65509804-592f-45db-9c01-4f9672da4e3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:22.868149 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ab5261-2483-40c0-b5d8-6eda15739c1e None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:22.868600 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ab5261-2483-40c0-b5d8-6eda15739c1e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:22.872684 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ab5261-2483-40c0-b5d8-6eda15739c1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:22.895711 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-e4ab5261-2483-40c0-b5d8-6eda15739c1e None None] MFA Rules not processed for user `d972ab782b3f4d2aba575314f16dafc7`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:03:22.897066 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e4ab5261-2483-40c0-b5d8-6eda15739c1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:22.979159 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:03:23.005843 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 817/1645] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:03:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 10 13:03:23.067779 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 829/1646] 10.222.0.27 () {58 vars in 1057 bytes} [Fri Jun 10 13:03:22 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:03:23.069511 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65509804-592f-45db-9c01-4f9672da4e3b None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:23.083574 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65509804-592f-45db-9c01-4f9672da4e3b None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:23.087241 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ab5261-2483-40c0-b5d8-6eda15739c1e None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:23.088679 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e4ab5261-2483-40c0-b5d8-6eda15739c1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:23.089724 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9819f59-21bf-4cf1-8017-18fa97901819 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd972ab782b3f4d2aba575314f16dafc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:23.091775 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9819f59-21bf-4cf1-8017-18fa97901819 service placement] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:23.092306 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9819f59-21bf-4cf1-8017-18fa97901819 service placement] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:23.093003 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9819f59-21bf-4cf1-8017-18fa97901819 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:23.095695 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d9819f59-21bf-4cf1-8017-18fa97901819 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:23.156301 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9819f59-21bf-4cf1-8017-18fa97901819 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:23.157919 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9819f59-21bf-4cf1-8017-18fa97901819 service placement] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:23.162776 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:03:23.187238 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 818/1647] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:03:23 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:03:23.211468 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ab5261-2483-40c0-b5d8-6eda15739c1e None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:23.216125 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1487bf-f3b1-4d86-8a9a-34629c25c062 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:23.216788 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1487bf-f3b1-4d86-8a9a-34629c25c062 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:23.217076 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1487bf-f3b1-4d86-8a9a-34629c25c062 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:23.217407 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1487bf-f3b1-4d86-8a9a-34629c25c062 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:23.229353 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1487bf-f3b1-4d86-8a9a-34629c25c062 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:23.230798 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1487bf-f3b1-4d86-8a9a-34629c25c062 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:23.245628 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 830/1648] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:03:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 10 13:03:24.264357 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9819f59-21bf-4cf1-8017-18fa97901819 service placement] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:24.265919 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d9819f59-21bf-4cf1-8017-18fa97901819 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:24.297159 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9819f59-21bf-4cf1-8017-18fa97901819 service placement] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:24.318917 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86daccd8-bf3d-4c57-8c27-64082c46ae4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:24.319957 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86daccd8-bf3d-4c57-8c27-64082c46ae4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:24.320241 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86daccd8-bf3d-4c57-8c27-64082c46ae4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:24.320513 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86daccd8-bf3d-4c57-8c27-64082c46ae4f admin admin] PATH_INFO: `/v3/projects/6cfc84b3dd644da3b65856da2defb668` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:24.325836 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86daccd8-bf3d-4c57-8c27-64082c46ae4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6cfc84b3dd644da3b65856da2defb668)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:24.328340 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86daccd8-bf3d-4c57-8c27-64082c46ae4f admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:24.491371 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-86daccd8-bf3d-4c57-8c27-64082c46ae4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6cfc84b3dd644da3b65856da2defb668', 'request_id': 'req-86daccd8-bf3d-4c57-8c27-64082c46ae4f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:03:24.503115 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-86daccd8-bf3d-4c57-8c27-64082c46ae4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6cfc84b3dd644da3b65856da2defb668', 'request_id': 'req-86daccd8-bf3d-4c57-8c27-64082c46ae4f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:03:24.518015 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-86daccd8-bf3d-4c57-8c27-64082c46ae4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6cfc84b3dd644da3b65856da2defb668', 'request_id': 'req-86daccd8-bf3d-4c57-8c27-64082c46ae4f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:03:24.658768 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-86daccd8-bf3d-4c57-8c27-64082c46ae4f admin admin] The token cache is being invalidate because project 6cfc84b3dd644da3b65856da2defb668 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:03:24.659201 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-86daccd8-bf3d-4c57-8c27-64082c46ae4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:03:24.661401 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-86daccd8-bf3d-4c57-8c27-64082c46ae4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:03:24.663060 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 819/1649] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:03:24 2022] DELETE /identity/v3/projects/6cfc84b3dd644da3b65856da2defb668 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:03:24.667059 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1487bf-f3b1-4d86-8a9a-34629c25c062 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:24.678429 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1487bf-f3b1-4d86-8a9a-34629c25c062 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:24.755049 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86daccd8-bf3d-4c57-8c27-64082c46ae4f admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:24.756628 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-86daccd8-bf3d-4c57-8c27-64082c46ae4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:24.801497 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a63d8c-cb0f-4bf6-a78e-790618ee03c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:24.801497 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a63d8c-cb0f-4bf6-a78e-790618ee03c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:24.801973 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a63d8c-cb0f-4bf6-a78e-790618ee03c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:24.802464 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a63d8c-cb0f-4bf6-a78e-790618ee03c2 admin admin] PATH_INFO: `/v3/users/979f3e87ea42484d93c6340dba790e83` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:24.803971 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a63d8c-cb0f-4bf6-a78e-790618ee03c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=979f3e87ea42484d93c6340dba790e83)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:24.806128 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a63d8c-cb0f-4bf6-a78e-790618ee03c2 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:24.916348 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86daccd8-bf3d-4c57-8c27-64082c46ae4f admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:24.938634 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b4d26e-5c5f-4672-ac25-a526e7c3ff91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:24.938634 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b4d26e-5c5f-4672-ac25-a526e7c3ff91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:24.938634 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b4d26e-5c5f-4672-ac25-a526e7c3ff91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:24.938920 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b4d26e-5c5f-4672-ac25-a526e7c3ff91 admin admin] PATH_INFO: `/v3/users/655b6e90e39c46718573d84d54e77219` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:24.943570 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b4d26e-5c5f-4672-ac25-a526e7c3ff91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=655b6e90e39c46718573d84d54e77219)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:24.944996 dv-cinder-master-3-smb devstack@keystone.service[80238]: WARNING py.warnings [None req-84b4d26e-5c5f-4672-ac25-a526e7c3ff91 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 10 13:03:24.944996 dv-cinder-master-3-smb devstack@keystone.service[80238]: warnings.warn(msg) Jun 10 13:03:24.944996 dv-cinder-master-3-smb devstack@keystone.service[80238]: Jun 10 13:03:24.947408 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b4d26e-5c5f-4672-ac25-a526e7c3ff91 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:25.037786 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-98a63d8c-cb0f-4bf6-a78e-790618ee03c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '979f3e87ea42484d93c6340dba790e83'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:03:25.046747 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-98a63d8c-cb0f-4bf6-a78e-790618ee03c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '979f3e87ea42484d93c6340dba790e83'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:03:25.134212 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-98a63d8c-cb0f-4bf6-a78e-790618ee03c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '979f3e87ea42484d93c6340dba790e83'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:03:25.136324 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-98a63d8c-cb0f-4bf6-a78e-790618ee03c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '979f3e87ea42484d93c6340dba790e83'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:03:25.164296 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-98a63d8c-cb0f-4bf6-a78e-790618ee03c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '979f3e87ea42484d93c6340dba790e83'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:03:25.176575 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 831/1650] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:03:24 2022] DELETE /identity/v3/users/979f3e87ea42484d93c6340dba790e83 => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 10 13:03:25.283475 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-84b4d26e-5c5f-4672-ac25-a526e7c3ff91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '655b6e90e39c46718573d84d54e77219'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:03:25.304726 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-84b4d26e-5c5f-4672-ac25-a526e7c3ff91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '655b6e90e39c46718573d84d54e77219'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:03:25.441498 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-84b4d26e-5c5f-4672-ac25-a526e7c3ff91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '655b6e90e39c46718573d84d54e77219'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:03:25.442565 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-84b4d26e-5c5f-4672-ac25-a526e7c3ff91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '655b6e90e39c46718573d84d54e77219'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:03:25.459152 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:03:25.472322 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-84b4d26e-5c5f-4672-ac25-a526e7c3ff91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '655b6e90e39c46718573d84d54e77219'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:03:25.483898 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 820/1651] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:03:24 2022] DELETE /identity/v3/users/655b6e90e39c46718573d84d54e77219 => generated 0 bytes in 729 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:03:25.489636 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8556f1-38be-446b-b400-4b345571aae2 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:25.489932 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8556f1-38be-446b-b400-4b345571aae2 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:25.490215 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8556f1-38be-446b-b400-4b345571aae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:25.535902 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-ad8556f1-38be-446b-b400-4b345571aae2 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:03:25.537639 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ad8556f1-38be-446b-b400-4b345571aae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:25.651640 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:03:25.679956 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 832/1652] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:03:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:03:25.720868 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b4d26e-5c5f-4672-ac25-a526e7c3ff91 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:25.723107 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-84b4d26e-5c5f-4672-ac25-a526e7c3ff91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:25.878916 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b4d26e-5c5f-4672-ac25-a526e7c3ff91 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:25.900903 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7bc00d-cd36-42c0-b4d1-7eb06d63abbf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:25.901935 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7bc00d-cd36-42c0-b4d1-7eb06d63abbf service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:25.902349 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7bc00d-cd36-42c0-b4d1-7eb06d63abbf service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:25.903788 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7bc00d-cd36-42c0-b4d1-7eb06d63abbf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:25.936023 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7bc00d-cd36-42c0-b4d1-7eb06d63abbf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:25.937826 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7bc00d-cd36-42c0-b4d1-7eb06d63abbf service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:26.011284 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 821/1653] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:03:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:03:26.915770 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8556f1-38be-446b-b400-4b345571aae2 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:26.921346 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ad8556f1-38be-446b-b400-4b345571aae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:26.950125 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8556f1-38be-446b-b400-4b345571aae2 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:26.964188 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88368966-853a-4c50-b8c7-a064c3a51c38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:26.966554 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88368966-853a-4c50-b8c7-a064c3a51c38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:26.967194 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88368966-853a-4c50-b8c7-a064c3a51c38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:26.969017 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88368966-853a-4c50-b8c7-a064c3a51c38 admin admin] PATH_INFO: `/v3/projects/03b2274321df4dbcaa54ce9ab1cbce64` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:26.976700 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88368966-853a-4c50-b8c7-a064c3a51c38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=03b2274321df4dbcaa54ce9ab1cbce64)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:26.978671 dv-cinder-master-3-smb devstack@keystone.service[80237]: WARNING py.warnings [None req-88368966-853a-4c50-b8c7-a064c3a51c38 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 10 13:03:26.978671 dv-cinder-master-3-smb devstack@keystone.service[80237]: warnings.warn(msg) Jun 10 13:03:26.978671 dv-cinder-master-3-smb devstack@keystone.service[80237]: Jun 10 13:03:26.980267 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88368966-853a-4c50-b8c7-a064c3a51c38 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:27.198036 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-88368966-853a-4c50-b8c7-a064c3a51c38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03b2274321df4dbcaa54ce9ab1cbce64', 'request_id': 'req-88368966-853a-4c50-b8c7-a064c3a51c38'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:03:27.217042 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-88368966-853a-4c50-b8c7-a064c3a51c38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03b2274321df4dbcaa54ce9ab1cbce64', 'request_id': 'req-88368966-853a-4c50-b8c7-a064c3a51c38'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:03:27.244065 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-88368966-853a-4c50-b8c7-a064c3a51c38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03b2274321df4dbcaa54ce9ab1cbce64', 'request_id': 'req-88368966-853a-4c50-b8c7-a064c3a51c38'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:03:27.346187 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-88368966-853a-4c50-b8c7-a064c3a51c38 admin admin] The token cache is being invalidate because project 03b2274321df4dbcaa54ce9ab1cbce64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:03:27.346736 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-88368966-853a-4c50-b8c7-a064c3a51c38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:03:27.347555 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-88368966-853a-4c50-b8c7-a064c3a51c38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:03:27.349688 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 833/1654] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:03:26 2022] DELETE /identity/v3/projects/03b2274321df4dbcaa54ce9ab1cbce64 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:03:27.355303 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cde5956-b343-4bc0-8dd6-46183bb66943 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:27.357062 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cde5956-b343-4bc0-8dd6-46183bb66943 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:27.357795 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cde5956-b343-4bc0-8dd6-46183bb66943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:27.415818 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-9cde5956-b343-4bc0-8dd6-46183bb66943 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:03:27.423074 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-9cde5956-b343-4bc0-8dd6-46183bb66943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:27.687636 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 822/1655] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:03:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:03:27.696599 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3f1324-441b-44c3-8624-adb5fad292ac None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:27.696893 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3f1324-441b-44c3-8624-adb5fad292ac None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:27.697179 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3f1324-441b-44c3-8624-adb5fad292ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:27.732577 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-1e3f1324-441b-44c3-8624-adb5fad292ac None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:03:27.733865 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1e3f1324-441b-44c3-8624-adb5fad292ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:27.778192 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 834/1656] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:03:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:03:27.781428 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cde5956-b343-4bc0-8dd6-46183bb66943 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:27.795711 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cde5956-b343-4bc0-8dd6-46183bb66943 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:27.832413 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4398a34c-c9f6-4b0d-95ca-f2923ec14e4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:27.833513 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4398a34c-c9f6-4b0d-95ca-f2923ec14e4d admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:27.834006 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4398a34c-c9f6-4b0d-95ca-f2923ec14e4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:27.835037 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4398a34c-c9f6-4b0d-95ca-f2923ec14e4d admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:27.836003 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4398a34c-c9f6-4b0d-95ca-f2923ec14e4d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 13:03:27.836513 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4398a34c-c9f6-4b0d-95ca-f2923ec14e4d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:27.838366 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4398a34c-c9f6-4b0d-95ca-f2923ec14e4d admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:27.903720 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 823/1657] 10.222.0.27 () {58 vars in 1152 bytes} [Fri Jun 10 13:03:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 13:03:27.904680 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3f1324-441b-44c3-8624-adb5fad292ac None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:27.922404 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3f1324-441b-44c3-8624-adb5fad292ac None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:27.930223 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bd820b-355a-4a27-b1c7-bc0f2ef8a07c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:27.930971 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bd820b-355a-4a27-b1c7-bc0f2ef8a07c admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:27.931414 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bd820b-355a-4a27-b1c7-bc0f2ef8a07c admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:27.932077 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bd820b-355a-4a27-b1c7-bc0f2ef8a07c admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:27.932929 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bd820b-355a-4a27-b1c7-bc0f2ef8a07c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:27.937549 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bd820b-355a-4a27-b1c7-bc0f2ef8a07c admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:28.154635 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 835/1658] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:03:27 2022] POST /identity/v3/projects => generated 363 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:03:28.164733 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4398a34c-c9f6-4b0d-95ca-f2923ec14e4d admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:28.165457 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bd820b-355a-4a27-b1c7-bc0f2ef8a07c admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:28.168562 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-72bd820b-355a-4a27-b1c7-bc0f2ef8a07c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:28.175212 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4398a34c-c9f6-4b0d-95ca-f2923ec14e4d admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:28.331301 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bd820b-355a-4a27-b1c7-bc0f2ef8a07c admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:28.335020 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971d4142-ba06-469f-8eeb-e749636e5454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:28.336368 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971d4142-ba06-469f-8eeb-e749636e5454 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:28.336854 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971d4142-ba06-469f-8eeb-e749636e5454 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:28.337147 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971d4142-ba06-469f-8eeb-e749636e5454 admin admin] PATH_INFO: `/v3/projects/7334d4dfa33941458c237db1a6c32809` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:28.338440 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971d4142-ba06-469f-8eeb-e749636e5454 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7334d4dfa33941458c237db1a6c32809)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:28.341764 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971d4142-ba06-469f-8eeb-e749636e5454 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:28.407218 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d29a3b8-75f7-45aa-8c05-461754b46940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:28.408444 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d29a3b8-75f7-45aa-8c05-461754b46940 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:28.408870 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d29a3b8-75f7-45aa-8c05-461754b46940 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:28.409307 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d29a3b8-75f7-45aa-8c05-461754b46940 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:28.410145 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d29a3b8-75f7-45aa-8c05-461754b46940 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:28.412985 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d29a3b8-75f7-45aa-8c05-461754b46940 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:28.503793 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:03:28.511017 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 824/1659] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:03:28 2022] POST /identity/v3/users => generated 282 bytes in 347 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:03:28.512446 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbe80d3-cd1f-4e3d-b58c-f1ce4d72be0b None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:28.512867 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbe80d3-cd1f-4e3d-b58c-f1ce4d72be0b None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:28.513157 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbe80d3-cd1f-4e3d-b58c-f1ce4d72be0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:28.513451 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-971d4142-ba06-469f-8eeb-e749636e5454 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7334d4dfa33941458c237db1a6c32809', 'request_id': 'req-971d4142-ba06-469f-8eeb-e749636e5454'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:03:28.527822 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-971d4142-ba06-469f-8eeb-e749636e5454 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7334d4dfa33941458c237db1a6c32809', 'request_id': 'req-971d4142-ba06-469f-8eeb-e749636e5454'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:03:28.540534 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-971d4142-ba06-469f-8eeb-e749636e5454 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7334d4dfa33941458c237db1a6c32809', 'request_id': 'req-971d4142-ba06-469f-8eeb-e749636e5454'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:03:28.561100 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-1bbe80d3-cd1f-4e3d-b58c-f1ce4d72be0b None None] MFA Rules not processed for user `da66362070694e91bbe2cfbbd161b8a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:03:28.562619 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-1bbe80d3-cd1f-4e3d-b58c-f1ce4d72be0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:28.581300 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:03:28.667003 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-971d4142-ba06-469f-8eeb-e749636e5454 admin admin] The token cache is being invalidate because project 7334d4dfa33941458c237db1a6c32809 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:03:28.667522 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-971d4142-ba06-469f-8eeb-e749636e5454 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:03:28.669505 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-971d4142-ba06-469f-8eeb-e749636e5454 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:03:28.671557 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 836/1660] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:03:28 2022] DELETE /identity/v3/projects/7334d4dfa33941458c237db1a6c32809 => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:03:28.672571 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971d4142-ba06-469f-8eeb-e749636e5454 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:28.689589 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-971d4142-ba06-469f-8eeb-e749636e5454 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:28.768311 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 825/1661] 10.222.0.27 () {58 vars in 1009 bytes} [Fri Jun 10 13:03:28 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:03:28.770040 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e3bad1-8b93-4b52-9763-f065a618b17b None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:28.782460 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e3bad1-8b93-4b52-9763-f065a618b17b None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:28.782745 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e3bad1-8b93-4b52-9763-f065a618b17b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:28.833271 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-e3e3bad1-8b93-4b52-9763-f065a618b17b None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:03:28.838410 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e3e3bad1-8b93-4b52-9763-f065a618b17b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:28.929715 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971d4142-ba06-469f-8eeb-e749636e5454 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:28.958036 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac3f19f-5c19-426a-9131-e362d7317cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:28.958795 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac3f19f-5c19-426a-9131-e362d7317cab admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:28.959040 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac3f19f-5c19-426a-9131-e362d7317cab admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:28.959321 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac3f19f-5c19-426a-9131-e362d7317cab admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:28.960713 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac3f19f-5c19-426a-9131-e362d7317cab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:28.962616 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac3f19f-5c19-426a-9131-e362d7317cab admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:28.991906 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 837/1662] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:03:28 2022] GET /identity/v3/roles => generated 1420 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:03:28.996906 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac3f19f-5c19-426a-9131-e362d7317cab admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:29.024310 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac3f19f-5c19-426a-9131-e362d7317cab admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:29.042781 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4882bf-6cb3-44d6-a329-aabe070a2074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:29.045142 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4882bf-6cb3-44d6-a329-aabe070a2074 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:29.045694 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4882bf-6cb3-44d6-a329-aabe070a2074 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:29.046631 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4882bf-6cb3-44d6-a329-aabe070a2074 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:29.047813 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4882bf-6cb3-44d6-a329-aabe070a2074 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:29.051235 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4882bf-6cb3-44d6-a329-aabe070a2074 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:29.095202 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 838/1663] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:03:28 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:03:29.097892 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4882bf-6cb3-44d6-a329-aabe070a2074 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:29.116144 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4882bf-6cb3-44d6-a329-aabe070a2074 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:29.157833 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2618018-81ec-4817-98c8-7ad441ab7282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:29.158762 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2618018-81ec-4817-98c8-7ad441ab7282 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:29.159052 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2618018-81ec-4817-98c8-7ad441ab7282 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:29.159337 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2618018-81ec-4817-98c8-7ad441ab7282 admin admin] PATH_INFO: `/v3/projects/4e51c012a9b3438fa8cf3af68950bdec/users/7a5c6f2799ac468c9db5569923e6f3a3/roles/43106c664e1b40de984c0c62701149c8` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:29.224916 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2618018-81ec-4817-98c8-7ad441ab7282 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e51c012a9b3438fa8cf3af68950bdec, user_id=7a5c6f2799ac468c9db5569923e6f3a3, role_id=43106c664e1b40de984c0c62701149c8)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:29.234936 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2618018-81ec-4817-98c8-7ad441ab7282 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:29.354305 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 826/1664] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:03:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 585 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:03:29.361103 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e3bad1-8b93-4b52-9763-f065a618b17b None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:29.368987 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e3e3bad1-8b93-4b52-9763-f065a618b17b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:29.390886 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 839/1665] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:03:29 2022] PUT /identity/v3/projects/4e51c012a9b3438fa8cf3af68950bdec/users/7a5c6f2799ac468c9db5569923e6f3a3/roles/43106c664e1b40de984c0c62701149c8 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:03:29.390886 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6694c9f0-a827-495d-9d44-2cde4b1739df None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:29.390886 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6694c9f0-a827-495d-9d44-2cde4b1739df None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:29.390886 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6694c9f0-a827-495d-9d44-2cde4b1739df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:29.455596 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-6694c9f0-a827-495d-9d44-2cde4b1739df None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:03:29.460462 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6694c9f0-a827-495d-9d44-2cde4b1739df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:29.541187 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e3bad1-8b93-4b52-9763-f065a618b17b None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:29.644996 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 840/1666] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:03:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:03:29.648734 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:03:29.649794 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f589c1-a20a-4a2b-b550-738b859dfabb None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:29.650456 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f589c1-a20a-4a2b-b550-738b859dfabb None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:29.651879 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f589c1-a20a-4a2b-b550-738b859dfabb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:29.666383 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc3297d-1a13-493e-ac33-5869ae9a561a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb63b9cd74bf438aa25b72cef8337fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:29.667225 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc3297d-1a13-493e-ac33-5869ae9a561a service swift] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:29.667514 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc3297d-1a13-493e-ac33-5869ae9a561a service swift] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:29.667795 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc3297d-1a13-493e-ac33-5869ae9a561a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:29.735067 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-28f589c1-a20a-4a2b-b550-738b859dfabb None None] MFA Rules not processed for user `7a5c6f2799ac468c9db5569923e6f3a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:03:29.736576 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-28f589c1-a20a-4a2b-b550-738b859dfabb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:29.801754 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:DELETE=4,keystone:SELECT=94,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:03:29.835852 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc3297d-1a13-493e-ac33-5869ae9a561a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:29.837512 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc3297d-1a13-493e-ac33-5869ae9a561a service swift] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:29.879524 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 827/1667] 10.222.0.27 () {60 vars in 1384 bytes} [Fri Jun 10 13:03:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 521 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 10 13:03:29.881823 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc3297d-1a13-493e-ac33-5869ae9a561a service swift] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:29.917780 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 841/1668] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:03:29 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:03:29.919336 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7972852d-912e-4418-93b2-a36f6a5c0562 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:29.919819 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7972852d-912e-4418-93b2-a36f6a5c0562 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:29.920132 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7972852d-912e-4418-93b2-a36f6a5c0562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:29.925190 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc3297d-1a13-493e-ac33-5869ae9a561a service swift] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:29.953185 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56feb5df-6d59-44ef-bef3-f230b41fa778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:29.954348 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56feb5df-6d59-44ef-bef3-f230b41fa778 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:29.954846 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56feb5df-6d59-44ef-bef3-f230b41fa778 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:29.955379 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56feb5df-6d59-44ef-bef3-f230b41fa778 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:29.955976 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56feb5df-6d59-44ef-bef3-f230b41fa778 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 13:03:29.956291 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56feb5df-6d59-44ef-bef3-f230b41fa778 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:29.957973 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56feb5df-6d59-44ef-bef3-f230b41fa778 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:30.001093 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 828/1669] 10.222.0.27 () {58 vars in 1152 bytes} [Fri Jun 10 13:03:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 13:03:30.002200 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56feb5df-6d59-44ef-bef3-f230b41fa778 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:30.007599 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-7972852d-912e-4418-93b2-a36f6a5c0562 None None] MFA Rules not processed for user `634e6116f688494c9e11d59479f94a43`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:03:30.008459 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-7972852d-912e-4418-93b2-a36f6a5c0562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:30.033205 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56feb5df-6d59-44ef-bef3-f230b41fa778 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:30.053251 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d698a1-297a-4032-8292-706b9d551b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:30.053937 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d698a1-297a-4032-8292-706b9d551b23 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:30.054224 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d698a1-297a-4032-8292-706b9d551b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:30.054687 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d698a1-297a-4032-8292-706b9d551b23 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:30.055398 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d698a1-297a-4032-8292-706b9d551b23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:30.058655 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d698a1-297a-4032-8292-706b9d551b23 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:30.205846 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 829/1670] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:03:30 2022] POST /identity/v3/projects => generated 365 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:03:30.205846 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d698a1-297a-4032-8292-706b9d551b23 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:30.264150 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 842/1671] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:03:29 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:03:30.265510 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7972852d-912e-4418-93b2-a36f6a5c0562 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:30.287996 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7972852d-912e-4418-93b2-a36f6a5c0562 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:30.384536 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d698a1-297a-4032-8292-706b9d551b23 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:30.399221 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d12d692-5f2f-42e7-9f6f-1e15f1228da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:30.409642 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d12d692-5f2f-42e7-9f6f-1e15f1228da4 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:30.411352 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d12d692-5f2f-42e7-9f6f-1e15f1228da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:30.412171 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d12d692-5f2f-42e7-9f6f-1e15f1228da4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:30.412997 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d12d692-5f2f-42e7-9f6f-1e15f1228da4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:30.424540 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d12d692-5f2f-42e7-9f6f-1e15f1228da4 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:30.462666 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55b7f82-b843-4a04-bcca-804ee8e1317a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:30.463371 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55b7f82-b843-4a04-bcca-804ee8e1317a admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:30.463736 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55b7f82-b843-4a04-bcca-804ee8e1317a admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:30.464072 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55b7f82-b843-4a04-bcca-804ee8e1317a admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:30.465448 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55b7f82-b843-4a04-bcca-804ee8e1317a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:30.473968 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55b7f82-b843-4a04-bcca-804ee8e1317a admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:30.585899 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 843/1672] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:03:30 2022] POST /identity/v3/users => generated 289 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:03:30.588490 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55b7f82-b843-4a04-bcca-804ee8e1317a admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:30.595048 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d55b7f82-b843-4a04-bcca-804ee8e1317a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:30.600954 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 830/1673] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:03:30 2022] POST /identity/v3/projects => generated 373 bytes in 395 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:03:30.600954 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d12d692-5f2f-42e7-9f6f-1e15f1228da4 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:30.786706 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:03:30.866810 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:03:30.877231 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55b7f82-b843-4a04-bcca-804ee8e1317a admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:30.925305 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d12d692-5f2f-42e7-9f6f-1e15f1228da4 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:30.947137 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21eb36bf-97c1-4dc4-90e8-a42f5c7d8b75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:30.948042 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eb36bf-97c1-4dc4-90e8-a42f5c7d8b75 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:30.948336 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eb36bf-97c1-4dc4-90e8-a42f5c7d8b75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:30.948676 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eb36bf-97c1-4dc4-90e8-a42f5c7d8b75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:30.975180 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21eb36bf-97c1-4dc4-90e8-a42f5c7d8b75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:31.003449 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21eb36bf-97c1-4dc4-90e8-a42f5c7d8b75 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:31.037179 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 831/1674] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:03:30 2022] GET /identity/v3/roles => generated 1420 bytes in 438 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:03:31.038412 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21eb36bf-97c1-4dc4-90e8-a42f5c7d8b75 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:31.082395 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21eb36bf-97c1-4dc4-90e8-a42f5c7d8b75 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:31.111195 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b33ca73-8994-4481-a51a-ef505977aee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:31.112099 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b33ca73-8994-4481-a51a-ef505977aee6 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:31.115449 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b33ca73-8994-4481-a51a-ef505977aee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:31.115886 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b33ca73-8994-4481-a51a-ef505977aee6 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:31.116880 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b33ca73-8994-4481-a51a-ef505977aee6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:31.123465 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b33ca73-8994-4481-a51a-ef505977aee6 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:31.197935 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88315e6e-0e09-4d3f-816a-23c274498f94 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bd6caa461e84100a3d5fe5af5d80f3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:31.199912 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88315e6e-0e09-4d3f-816a-23c274498f94 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:31.200394 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88315e6e-0e09-4d3f-816a-23c274498f94 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:31.201050 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88315e6e-0e09-4d3f-816a-23c274498f94 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:31.224869 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 832/1675] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:03:31 2022] POST /identity/v3/users => generated 287 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:03:31.226489 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b33ca73-8994-4481-a51a-ef505977aee6 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:31.284978 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b33ca73-8994-4481-a51a-ef505977aee6 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:31.312862 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9c5731-1d2a-4938-b73c-49904af25ab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:31.314011 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9c5731-1d2a-4938-b73c-49904af25ab8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:31.317127 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9c5731-1d2a-4938-b73c-49904af25ab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:31.319966 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9c5731-1d2a-4938-b73c-49904af25ab8 admin admin] PATH_INFO: `/v3/projects/a3334230920d4b8c8b098837ea9bb4b6/users/a6786c83c72d410f99ad5521b523cb26/roles/b43d5a8247e244869c2c70f986f42e8a` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:31.365897 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9c5731-1d2a-4938-b73c-49904af25ab8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3334230920d4b8c8b098837ea9bb4b6, user_id=a6786c83c72d410f99ad5521b523cb26, role_id=b43d5a8247e244869c2c70f986f42e8a)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:31.367131 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88315e6e-0e09-4d3f-816a-23c274498f94 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:31.369507 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9c5731-1d2a-4938-b73c-49904af25ab8 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:31.370627 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88315e6e-0e09-4d3f-816a-23c274498f94 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:31.405715 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 844/1676] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:03:30 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 817 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:03:31.407654 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88315e6e-0e09-4d3f-816a-23c274498f94 service nova] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:31.459355 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88315e6e-0e09-4d3f-816a-23c274498f94 service nova] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:31.475007 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b416317-b711-4c96-97d3-ea096f10e55f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:31.478405 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b416317-b711-4c96-97d3-ea096f10e55f admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:31.479117 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b416317-b711-4c96-97d3-ea096f10e55f admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:31.480945 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b416317-b711-4c96-97d3-ea096f10e55f admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:31.483495 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b416317-b711-4c96-97d3-ea096f10e55f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:31.486014 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 833/1677] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:03:31 2022] PUT /identity/v3/projects/a3334230920d4b8c8b098837ea9bb4b6/users/a6786c83c72d410f99ad5521b523cb26/roles/b43d5a8247e244869c2c70f986f42e8a => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:03:31.487053 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9c5731-1d2a-4938-b73c-49904af25ab8 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:31.489500 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b416317-b711-4c96-97d3-ea096f10e55f admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:31.523484 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 845/1678] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:03:31 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:03:31.528925 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b416317-b711-4c96-97d3-ea096f10e55f admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:31.553210 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b416317-b711-4c96-97d3-ea096f10e55f admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:31.718408 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9c5731-1d2a-4938-b73c-49904af25ab8 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:31.737874 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34097668-7938-4289-ab6a-a4c769fc4fe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:31.738703 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34097668-7938-4289-ab6a-a4c769fc4fe4 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:31.739046 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34097668-7938-4289-ab6a-a4c769fc4fe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:31.739383 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34097668-7938-4289-ab6a-a4c769fc4fe4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:31.740230 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34097668-7938-4289-ab6a-a4c769fc4fe4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:31.741984 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34097668-7938-4289-ab6a-a4c769fc4fe4 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:31.776522 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9170b227-194e-4b8d-b89a-c37c626793e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:31.777196 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9170b227-194e-4b8d-b89a-c37c626793e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:31.778029 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9170b227-194e-4b8d-b89a-c37c626793e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:31.778465 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9170b227-194e-4b8d-b89a-c37c626793e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:31.779090 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9170b227-194e-4b8d-b89a-c37c626793e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:31.782748 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9170b227-194e-4b8d-b89a-c37c626793e2 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:31.792045 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 834/1679] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:03:31 2022] GET /identity/v3/roles => generated 1420 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:03:31.802830 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34097668-7938-4289-ab6a-a4c769fc4fe4 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:31.836919 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34097668-7938-4289-ab6a-a4c769fc4fe4 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:31.839099 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 846/1680] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:03:31 2022] GET /identity/v3/roles => generated 1420 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 10 13:03:31.843563 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9170b227-194e-4b8d-b89a-c37c626793e2 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:31.846524 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4967dcd3-6503-4cd0-8a43-91be468a9f36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:31.848990 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4967dcd3-6503-4cd0-8a43-91be468a9f36 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:31.850400 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4967dcd3-6503-4cd0-8a43-91be468a9f36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:31.850853 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4967dcd3-6503-4cd0-8a43-91be468a9f36 admin admin] PATH_INFO: `/v3/projects/a3334230920d4b8c8b098837ea9bb4b6/users/a6786c83c72d410f99ad5521b523cb26/roles/b43d5a8247e244869c2c70f986f42e8a` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:31.863809 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4967dcd3-6503-4cd0-8a43-91be468a9f36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3334230920d4b8c8b098837ea9bb4b6, user_id=a6786c83c72d410f99ad5521b523cb26, role_id=b43d5a8247e244869c2c70f986f42e8a)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:31.870392 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:03:31.875819 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4967dcd3-6503-4cd0-8a43-91be468a9f36 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:31.879396 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9170b227-194e-4b8d-b89a-c37c626793e2 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:31.908495 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76851957-d373-4754-8dc1-e24c45148c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:31.909138 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 835/1681] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:03:31 2022] PUT /identity/v3/projects/a3334230920d4b8c8b098837ea9bb4b6/users/a6786c83c72d410f99ad5521b523cb26/roles/b43d5a8247e244869c2c70f986f42e8a => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:03:31.910223 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76851957-d373-4754-8dc1-e24c45148c6a admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:31.910638 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76851957-d373-4754-8dc1-e24c45148c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:31.911041 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76851957-d373-4754-8dc1-e24c45148c6a admin admin] PATH_INFO: `/v3/projects/5629f339b5554aeaab8bea8a33ac21a4/users/6f46e18c70724dada243fde343e20eb7/roles/43106c664e1b40de984c0c62701149c8` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:31.922658 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5482fb-942d-4233-abaa-7ab4bc4a90cf None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:31.923056 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5482fb-942d-4233-abaa-7ab4bc4a90cf None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:31.923438 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5482fb-942d-4233-abaa-7ab4bc4a90cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:31.989742 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76851957-d373-4754-8dc1-e24c45148c6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5629f339b5554aeaab8bea8a33ac21a4, user_id=6f46e18c70724dada243fde343e20eb7, role_id=43106c664e1b40de984c0c62701149c8)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:31.992482 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76851957-d373-4754-8dc1-e24c45148c6a admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:32.041074 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-fa5482fb-942d-4233-abaa-7ab4bc4a90cf None None] MFA Rules not processed for user `a6786c83c72d410f99ad5521b523cb26`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:03:32.042414 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-fa5482fb-942d-4233-abaa-7ab4bc4a90cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:32.125502 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 847/1682] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:03:31 2022] PUT /identity/v3/projects/5629f339b5554aeaab8bea8a33ac21a4/users/6f46e18c70724dada243fde343e20eb7/roles/43106c664e1b40de984c0c62701149c8 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:03:32.129520 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c57b1-5e2b-4b15-949c-150ab3144fa1 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:32.129846 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c57b1-5e2b-4b15-949c-150ab3144fa1 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:32.130132 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c57b1-5e2b-4b15-949c-150ab3144fa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:32.285137 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-867c57b1-5e2b-4b15-949c-150ab3144fa1 None None] MFA Rules not processed for user `6f46e18c70724dada243fde343e20eb7`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:03:32.285137 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-867c57b1-5e2b-4b15-949c-150ab3144fa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:32.314286 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 836/1683] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:03:31 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 393 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:03:32.332904 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5482fb-942d-4233-abaa-7ab4bc4a90cf None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:32.334391 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-fa5482fb-942d-4233-abaa-7ab4bc4a90cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:32.377188 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:03:32.405476 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 848/1684] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:03:32 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:03:32.408018 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867c57b1-5e2b-4b15-949c-150ab3144fa1 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:32.523847 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5482fb-942d-4233-abaa-7ab4bc4a90cf None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:32.642073 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de9ee88-db9f-4d6b-ad9d-fd85702e50dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6cc95a563b941ac92ab283aa0d5a5d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:32.643756 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de9ee88-db9f-4d6b-ad9d-fd85702e50dc service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:32.643756 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de9ee88-db9f-4d6b-ad9d-fd85702e50dc service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:32.643756 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de9ee88-db9f-4d6b-ad9d-fd85702e50dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:32.703453 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867c57b1-5e2b-4b15-949c-150ab3144fa1 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:32.736958 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fb5177-747f-4599-8b91-9a105c9d762e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:32.738579 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fb5177-747f-4599-8b91-9a105c9d762e admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:32.738869 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fb5177-747f-4599-8b91-9a105c9d762e admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:32.739239 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fb5177-747f-4599-8b91-9a105c9d762e admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:32.748678 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fb5177-747f-4599-8b91-9a105c9d762e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:32.753089 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fb5177-747f-4599-8b91-9a105c9d762e admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:32.805961 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de9ee88-db9f-4d6b-ad9d-fd85702e50dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:32.808256 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de9ee88-db9f-4d6b-ad9d-fd85702e50dc service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:32.846354 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 837/1685] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:03:32 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 514 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 10 13:03:32.919878 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 849/1686] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:03:32 2022] POST /identity/v3/projects => generated 371 bytes in 513 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:03:32.931556 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de9ee88-db9f-4d6b-ad9d-fd85702e50dc service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:32.945153 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de9ee88-db9f-4d6b-ad9d-fd85702e50dc service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:33.114005 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f07a80b-934c-43b9-ad97-c11826b09e86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:33.114711 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f07a80b-934c-43b9-ad97-c11826b09e86 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:33.115000 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f07a80b-934c-43b9-ad97-c11826b09e86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:33.115276 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f07a80b-934c-43b9-ad97-c11826b09e86 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:33.116011 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f07a80b-934c-43b9-ad97-c11826b09e86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:33.117954 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f07a80b-934c-43b9-ad97-c11826b09e86 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:33.289928 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 838/1687] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:03:32 2022] POST /identity/v3/users => generated 292 bytes in 363 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:03:33.291279 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fb5177-747f-4599-8b91-9a105c9d762e admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:33.312090 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fb5177-747f-4599-8b91-9a105c9d762e admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:33.334288 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330b54ee-0a47-47b1-97a7-f321bda21e9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:33.335654 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330b54ee-0a47-47b1-97a7-f321bda21e9d admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:33.336008 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330b54ee-0a47-47b1-97a7-f321bda21e9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:33.336286 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330b54ee-0a47-47b1-97a7-f321bda21e9d admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:33.337046 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330b54ee-0a47-47b1-97a7-f321bda21e9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:33.351305 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330b54ee-0a47-47b1-97a7-f321bda21e9d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:33.396794 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 850/1688] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:03:33 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 10 13:03:33.397856 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f07a80b-934c-43b9-ad97-c11826b09e86 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:33.416768 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f07a80b-934c-43b9-ad97-c11826b09e86 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:33.440206 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ff843f-cca4-483f-b8a3-dbafe26ac520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:33.440206 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ff843f-cca4-483f-b8a3-dbafe26ac520 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:33.440206 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ff843f-cca4-483f-b8a3-dbafe26ac520 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:33.440206 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ff843f-cca4-483f-b8a3-dbafe26ac520 admin admin] PATH_INFO: `/v3/projects/884ca11abc3540a79deffdea80c40232/users/8723b893a99a45879b2e28a54d5c7a28/roles/b43d5a8247e244869c2c70f986f42e8a` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:33.443213 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:03:33.475345 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ff843f-cca4-483f-b8a3-dbafe26ac520 admin admin] RBAC: Authorizing `identity:create_grant(project_id=884ca11abc3540a79deffdea80c40232, user_id=8723b893a99a45879b2e28a54d5c7a28, role_id=b43d5a8247e244869c2c70f986f42e8a)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:33.478210 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ff843f-cca4-483f-b8a3-dbafe26ac520 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:33.623498 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 839/1689] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:03:33 2022] PUT /identity/v3/projects/884ca11abc3540a79deffdea80c40232/users/8723b893a99a45879b2e28a54d5c7a28/roles/b43d5a8247e244869c2c70f986f42e8a => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:03:33.628678 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330b54ee-0a47-47b1-97a7-f321bda21e9d admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:33.639901 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330b54ee-0a47-47b1-97a7-f321bda21e9d admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:33.765609 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:03:33.802611 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5614bba2-887f-4fd3-ad86-951f339d3d6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:33.803970 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5614bba2-887f-4fd3-ad86-951f339d3d6f admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:33.804092 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5614bba2-887f-4fd3-ad86-951f339d3d6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:33.804650 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5614bba2-887f-4fd3-ad86-951f339d3d6f admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:33.805917 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5614bba2-887f-4fd3-ad86-951f339d3d6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:33.812365 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5614bba2-887f-4fd3-ad86-951f339d3d6f admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:33.834513 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 851/1690] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:03:33 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:03:33.838306 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ff843f-cca4-483f-b8a3-dbafe26ac520 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:33.853428 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ff843f-cca4-483f-b8a3-dbafe26ac520 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:33.884199 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f74323f-c4a5-4ef8-8c05-970464b2d4e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:33.886312 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f74323f-c4a5-4ef8-8c05-970464b2d4e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:33.888302 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f74323f-c4a5-4ef8-8c05-970464b2d4e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:33.888791 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f74323f-c4a5-4ef8-8c05-970464b2d4e6 admin admin] PATH_INFO: `/v3/projects/884ca11abc3540a79deffdea80c40232/users/8723b893a99a45879b2e28a54d5c7a28/roles/b43d5a8247e244869c2c70f986f42e8a` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:33.893365 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f74323f-c4a5-4ef8-8c05-970464b2d4e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=884ca11abc3540a79deffdea80c40232, user_id=8723b893a99a45879b2e28a54d5c7a28, role_id=b43d5a8247e244869c2c70f986f42e8a)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:33.899144 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f74323f-c4a5-4ef8-8c05-970464b2d4e6 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:33.958957 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 840/1691] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:03:33 2022] PUT /identity/v3/projects/884ca11abc3540a79deffdea80c40232/users/8723b893a99a45879b2e28a54d5c7a28/roles/b43d5a8247e244869c2c70f986f42e8a => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:03:33.960902 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c1eb09-f84d-4f92-98f6-9183913f5cba None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:33.961214 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c1eb09-f84d-4f92-98f6-9183913f5cba None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:33.961570 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c1eb09-f84d-4f92-98f6-9183913f5cba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:34.025742 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-76c1eb09-f84d-4f92-98f6-9183913f5cba None None] MFA Rules not processed for user `8723b893a99a45879b2e28a54d5c7a28`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:03:34.026999 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-76c1eb09-f84d-4f92-98f6-9183913f5cba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:34.052699 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f74323f-c4a5-4ef8-8c05-970464b2d4e6 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:34.054781 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-3f74323f-c4a5-4ef8-8c05-970464b2d4e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:34.220026 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 852/1692] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:03:33 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:03:34.233181 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c1eb09-f84d-4f92-98f6-9183913f5cba None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:34.234608 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-76c1eb09-f84d-4f92-98f6-9183913f5cba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:34.286323 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f74323f-c4a5-4ef8-8c05-970464b2d4e6 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:34.295412 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0743539b-a2a3-448e-9cce-7a5d0c34d465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:34.296514 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0743539b-a2a3-448e-9cce-7a5d0c34d465 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:34.296916 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0743539b-a2a3-448e-9cce-7a5d0c34d465 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:34.297310 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0743539b-a2a3-448e-9cce-7a5d0c34d465 admin admin] PATH_INFO: `/v3/users/782a5e6223c84471b2442ee0ff0eacf1` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:34.298970 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0743539b-a2a3-448e-9cce-7a5d0c34d465 admin admin] RBAC: Authorizing `identity:delete_user(user_id=782a5e6223c84471b2442ee0ff0eacf1)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:34.303227 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0743539b-a2a3-448e-9cce-7a5d0c34d465 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:34.460462 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c1eb09-f84d-4f92-98f6-9183913f5cba None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:34.469661 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996ee0a6-a639-4cb4-83e4-aa897ef8101a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6cc95a563b941ac92ab283aa0d5a5d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:34.470747 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996ee0a6-a639-4cb4-83e4-aa897ef8101a service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:34.485844 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996ee0a6-a639-4cb4-83e4-aa897ef8101a service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:34.486009 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996ee0a6-a639-4cb4-83e4-aa897ef8101a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:34.532947 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996ee0a6-a639-4cb4-83e4-aa897ef8101a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:34.535445 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996ee0a6-a639-4cb4-83e4-aa897ef8101a service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:34.594796 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 853/1693] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:03:34 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 10 13:03:34.595427 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996ee0a6-a639-4cb4-83e4-aa897ef8101a service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:34.628097 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-0743539b-a2a3-448e-9cce-7a5d0c34d465 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '782a5e6223c84471b2442ee0ff0eacf1'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:03:34.661539 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996ee0a6-a639-4cb4-83e4-aa897ef8101a service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:34.678031 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252828f3-b424-491a-a7f9-cc18b14067e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:34.678136 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252828f3-b424-491a-a7f9-cc18b14067e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:34.678355 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252828f3-b424-491a-a7f9-cc18b14067e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:34.678850 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252828f3-b424-491a-a7f9-cc18b14067e1 admin admin] PATH_INFO: `/v3/users/7a5c6f2799ac468c9db5569923e6f3a3` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:34.684012 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252828f3-b424-491a-a7f9-cc18b14067e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a5c6f2799ac468c9db5569923e6f3a3)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:34.684462 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252828f3-b424-491a-a7f9-cc18b14067e1 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:34.685123 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-0743539b-a2a3-448e-9cce-7a5d0c34d465 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '782a5e6223c84471b2442ee0ff0eacf1'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:03:34.783508 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-0743539b-a2a3-448e-9cce-7a5d0c34d465 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '782a5e6223c84471b2442ee0ff0eacf1'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:03:34.784525 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-0743539b-a2a3-448e-9cce-7a5d0c34d465 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '782a5e6223c84471b2442ee0ff0eacf1'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:03:34.811091 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-0743539b-a2a3-448e-9cce-7a5d0c34d465 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '782a5e6223c84471b2442ee0ff0eacf1'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:03:34.876133 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 841/1694] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:03:34 2022] DELETE /identity/v3/users/782a5e6223c84471b2442ee0ff0eacf1 => generated 0 bytes in 824 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:03:34.880525 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977c335c-38ce-4e17-927f-9029c32fa56d None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:34.881121 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977c335c-38ce-4e17-927f-9029c32fa56d None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:34.881549 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977c335c-38ce-4e17-927f-9029c32fa56d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:34.938137 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-977c335c-38ce-4e17-927f-9029c32fa56d None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:03:34.939629 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-977c335c-38ce-4e17-927f-9029c32fa56d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:34.973085 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-252828f3-b424-491a-a7f9-cc18b14067e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a5c6f2799ac468c9db5569923e6f3a3'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:03:34.984828 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-252828f3-b424-491a-a7f9-cc18b14067e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a5c6f2799ac468c9db5569923e6f3a3'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:03:35.060322 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:03:35.088607 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 842/1695] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:03:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:03:35.088801 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-252828f3-b424-491a-a7f9-cc18b14067e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a5c6f2799ac468c9db5569923e6f3a3'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:03:35.089311 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-252828f3-b424-491a-a7f9-cc18b14067e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a5c6f2799ac468c9db5569923e6f3a3'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:03:35.090255 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977c335c-38ce-4e17-927f-9029c32fa56d None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:35.091089 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:03:35.093121 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-977c335c-38ce-4e17-927f-9029c32fa56d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:35.131260 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-252828f3-b424-491a-a7f9-cc18b14067e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a5c6f2799ac468c9db5569923e6f3a3'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:03:35.146391 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 854/1696] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:03:34 2022] DELETE /identity/v3/users/7a5c6f2799ac468c9db5569923e6f3a3 => generated 0 bytes in 551 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:03:35.147583 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a452493-70c6-4234-b5ff-f764cf8b67de None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:35.147900 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a452493-70c6-4234-b5ff-f764cf8b67de None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:35.148191 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a452493-70c6-4234-b5ff-f764cf8b67de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:35.179923 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977c335c-38ce-4e17-927f-9029c32fa56d None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:35.193893 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-1a452493-70c6-4234-b5ff-f764cf8b67de None None] MFA Rules not processed for user `bc4415461a974c24ad7105a97c7d915b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:03:35.195140 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1a452493-70c6-4234-b5ff-f764cf8b67de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:35.336221 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 855/1697] 10.222.0.36 () {58 vars in 1015 bytes} [Fri Jun 10 13:03:35 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:03:35.336221 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a452493-70c6-4234-b5ff-f764cf8b67de None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:35.336221 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1a452493-70c6-4234-b5ff-f764cf8b67de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:35.345793 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb4b029-45c7-45a4-8ef4-014f74379365 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6cc95a563b941ac92ab283aa0d5a5d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:35.346409 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb4b029-45c7-45a4-8ef4-014f74379365 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:35.346706 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb4b029-45c7-45a4-8ef4-014f74379365 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:35.346988 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb4b029-45c7-45a4-8ef4-014f74379365 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:35.354343 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a452493-70c6-4234-b5ff-f764cf8b67de None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:35.357593 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-cbb4b029-45c7-45a4-8ef4-014f74379365 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:35.385417 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7750ed4d-5222-43d1-81e1-e8ba72f005c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:35.386479 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7750ed4d-5222-43d1-81e1-e8ba72f005c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:35.386962 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7750ed4d-5222-43d1-81e1-e8ba72f005c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:35.387552 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7750ed4d-5222-43d1-81e1-e8ba72f005c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:35.390594 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-7750ed4d-5222-43d1-81e1-e8ba72f005c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:35.572162 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7750ed4d-5222-43d1-81e1-e8ba72f005c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:35.574684 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7750ed4d-5222-43d1-81e1-e8ba72f005c3 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:35.606903 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb4b029-45c7-45a4-8ef4-014f74379365 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:35.606903 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb4b029-45c7-45a4-8ef4-014f74379365 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:35.628463 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 856/1698] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:03:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:03:35.633196 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfbca1a-f650-402b-ac22-7ece39e8fecc None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:35.633472 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfbca1a-f650-402b-ac22-7ece39e8fecc None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:35.633753 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfbca1a-f650-402b-ac22-7ece39e8fecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:35.657956 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 843/1699] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:03:35 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 568 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:03:35.658162 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb4b029-45c7-45a4-8ef4-014f74379365 service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:35.667547 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-cbb4b029-45c7-45a4-8ef4-014f74379365 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:35.724327 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-9dfbca1a-f650-402b-ac22-7ece39e8fecc None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:03:35.724673 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb4b029-45c7-45a4-8ef4-014f74379365 service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:35.731774 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9dfbca1a-f650-402b-ac22-7ece39e8fecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:35.743837 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e63df00-bab7-4e8e-9ebc-d7704f1e8240 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:35.749866 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e63df00-bab7-4e8e-9ebc-d7704f1e8240 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:35.749866 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e63df00-bab7-4e8e-9ebc-d7704f1e8240 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:35.749866 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e63df00-bab7-4e8e-9ebc-d7704f1e8240 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:35.770726 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 857/1700] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:03:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:03:35.794274 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e63df00-bab7-4e8e-9ebc-d7704f1e8240 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:35.798559 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e63df00-bab7-4e8e-9ebc-d7704f1e8240 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:35.812311 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfbca1a-f650-402b-ac22-7ece39e8fecc None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:35.826840 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfbca1a-f650-402b-ac22-7ece39e8fecc None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:35.832152 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5226b3-c61a-4874-b676-ff289a5e716c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:35.833322 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5226b3-c61a-4874-b676-ff289a5e716c service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:35.833618 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5226b3-c61a-4874-b676-ff289a5e716c service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:35.833904 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5226b3-c61a-4874-b676-ff289a5e716c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:35.840595 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 844/1701] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:03:35 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:03:35.847067 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5226b3-c61a-4874-b676-ff289a5e716c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:35.849009 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5226b3-c61a-4874-b676-ff289a5e716c service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:35.862002 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 858/1702] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:03:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 10 13:03:36.833482 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f539a0d-b332-44b6-9885-cc0aaf95cb63 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:36.834075 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f539a0d-b332-44b6-9885-cc0aaf95cb63 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:36.834075 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f539a0d-b332-44b6-9885-cc0aaf95cb63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:36.923385 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-0f539a0d-b332-44b6-9885-cc0aaf95cb63 None None] MFA Rules not processed for user `c71b903e00214894b94b03b3401607a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:03:36.929157 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0f539a0d-b332-44b6-9885-cc0aaf95cb63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:37.046724 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:03:37.134334 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 845/1703] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:03:36 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:03:37.147768 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5226b3-c61a-4874-b676-ff289a5e716c service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:37.149324 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2a5226b3-c61a-4874-b676-ff289a5e716c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:37.323225 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:03:37.331163 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5226b3-c61a-4874-b676-ff289a5e716c service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:37.348508 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4194f6de-bd6d-446a-a1ef-7272631479f1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bd6caa461e84100a3d5fe5af5d80f3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:37.349255 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4194f6de-bd6d-446a-a1ef-7272631479f1 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:37.349558 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4194f6de-bd6d-446a-a1ef-7272631479f1 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:37.349841 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4194f6de-bd6d-446a-a1ef-7272631479f1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:37.368493 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4194f6de-bd6d-446a-a1ef-7272631479f1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:37.370138 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4194f6de-bd6d-446a-a1ef-7272631479f1 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:37.401487 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 859/1704] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:03:37 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:03:38.209574 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f539a0d-b332-44b6-9885-cc0aaf95cb63 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:38.211121 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0f539a0d-b332-44b6-9885-cc0aaf95cb63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:38.257697 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f539a0d-b332-44b6-9885-cc0aaf95cb63 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:38.283981 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1017887f-f6b3-4ece-9ad7-a978e9b16af9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:38.298172 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1017887f-f6b3-4ece-9ad7-a978e9b16af9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:38.298556 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1017887f-f6b3-4ece-9ad7-a978e9b16af9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:38.298849 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1017887f-f6b3-4ece-9ad7-a978e9b16af9 admin admin] PATH_INFO: `/v3/projects/2befb4791c1143218332a2419ec51324` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:38.310984 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1017887f-f6b3-4ece-9ad7-a978e9b16af9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2befb4791c1143218332a2419ec51324)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:38.320836 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1017887f-f6b3-4ece-9ad7-a978e9b16af9 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:38.513049 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-1017887f-f6b3-4ece-9ad7-a978e9b16af9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2befb4791c1143218332a2419ec51324', 'request_id': 'req-1017887f-f6b3-4ece-9ad7-a978e9b16af9'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:03:38.527511 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-1017887f-f6b3-4ece-9ad7-a978e9b16af9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2befb4791c1143218332a2419ec51324', 'request_id': 'req-1017887f-f6b3-4ece-9ad7-a978e9b16af9'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:03:38.539882 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-1017887f-f6b3-4ece-9ad7-a978e9b16af9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2befb4791c1143218332a2419ec51324', 'request_id': 'req-1017887f-f6b3-4ece-9ad7-a978e9b16af9'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:03:38.645480 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-1017887f-f6b3-4ece-9ad7-a978e9b16af9 admin admin] The token cache is being invalidate because project 2befb4791c1143218332a2419ec51324 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:03:38.645851 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-1017887f-f6b3-4ece-9ad7-a978e9b16af9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:03:38.648468 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-1017887f-f6b3-4ece-9ad7-a978e9b16af9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:03:38.650664 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 846/1705] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:03:38 2022] DELETE /identity/v3/projects/2befb4791c1143218332a2419ec51324 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:03:38.655852 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05482b8-4417-46bb-b2ca-cecc73ef6774 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:38.656820 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05482b8-4417-46bb-b2ca-cecc73ef6774 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:38.657253 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05482b8-4417-46bb-b2ca-cecc73ef6774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:38.722706 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-c05482b8-4417-46bb-b2ca-cecc73ef6774 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:03:38.735131 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c05482b8-4417-46bb-b2ca-cecc73ef6774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:38.946680 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 860/1706] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:03:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:03:38.955427 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7ac06c-49f1-4ece-85c3-abcee07c8a43 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:38.956000 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7ac06c-49f1-4ece-85c3-abcee07c8a43 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:38.956451 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7ac06c-49f1-4ece-85c3-abcee07c8a43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:39.001386 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-db7ac06c-49f1-4ece-85c3-abcee07c8a43 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:03:39.002753 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-db7ac06c-49f1-4ece-85c3-abcee07c8a43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:39.028036 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 847/1707] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:03:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:03:39.032175 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05482b8-4417-46bb-b2ca-cecc73ef6774 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:39.046372 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05482b8-4417-46bb-b2ca-cecc73ef6774 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:39.060667 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2948a156-0c82-4dae-b863-e73dda17eb76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:39.061845 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2948a156-0c82-4dae-b863-e73dda17eb76 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:39.062410 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2948a156-0c82-4dae-b863-e73dda17eb76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:39.063749 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2948a156-0c82-4dae-b863-e73dda17eb76 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:39.064674 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2948a156-0c82-4dae-b863-e73dda17eb76 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 13:03:39.082851 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2948a156-0c82-4dae-b863-e73dda17eb76 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:39.082851 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2948a156-0c82-4dae-b863-e73dda17eb76 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:39.131433 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7ac06c-49f1-4ece-85c3-abcee07c8a43 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:39.138379 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 861/1708] 10.222.0.27 () {58 vars in 1152 bytes} [Fri Jun 10 13:03:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 13:03:39.146603 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7ac06c-49f1-4ece-85c3-abcee07c8a43 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:39.176182 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfae864a-0cae-4831-a11a-732f73633505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:39.176182 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfae864a-0cae-4831-a11a-732f73633505 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:39.176182 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfae864a-0cae-4831-a11a-732f73633505 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:39.176182 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfae864a-0cae-4831-a11a-732f73633505 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:39.176637 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfae864a-0cae-4831-a11a-732f73633505 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:39.176637 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfae864a-0cae-4831-a11a-732f73633505 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:39.353395 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 848/1709] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:03:39 2022] POST /identity/v3/projects => generated 405 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:03:39.358511 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2948a156-0c82-4dae-b863-e73dda17eb76 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:39.373515 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2948a156-0c82-4dae-b863-e73dda17eb76 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:39.394482 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfae864a-0cae-4831-a11a-732f73633505 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:39.396359 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-cfae864a-0cae-4831-a11a-732f73633505 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:39.703772 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfae864a-0cae-4831-a11a-732f73633505 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:39.711391 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45aa655-8bdc-4c59-8dd1-d6eae00a5f13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:39.718906 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45aa655-8bdc-4c59-8dd1-d6eae00a5f13 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:39.718906 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45aa655-8bdc-4c59-8dd1-d6eae00a5f13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:39.718906 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45aa655-8bdc-4c59-8dd1-d6eae00a5f13 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:39.718906 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45aa655-8bdc-4c59-8dd1-d6eae00a5f13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:39.718906 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45aa655-8bdc-4c59-8dd1-d6eae00a5f13 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:39.735620 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1fbcd3-d30b-4f95-9138-b6cda99df77d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:39.738203 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1fbcd3-d30b-4f95-9138-b6cda99df77d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:39.741834 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1fbcd3-d30b-4f95-9138-b6cda99df77d admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:39.741834 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1fbcd3-d30b-4f95-9138-b6cda99df77d admin admin] PATH_INFO: `/v3/projects/4e51c012a9b3438fa8cf3af68950bdec` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:39.743084 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1fbcd3-d30b-4f95-9138-b6cda99df77d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e51c012a9b3438fa8cf3af68950bdec)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:39.745348 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1fbcd3-d30b-4f95-9138-b6cda99df77d admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:39.853498 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 862/1710] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:03:39 2022] POST /identity/v3/users => generated 303 bytes in 495 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:03:39.857316 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45aa655-8bdc-4c59-8dd1-d6eae00a5f13 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:39.908245 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45aa655-8bdc-4c59-8dd1-d6eae00a5f13 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:39.921613 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde0ff8a-c8bf-43ff-9b4e-3d8050932fd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:39.922510 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde0ff8a-c8bf-43ff-9b4e-3d8050932fd0 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:39.925525 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde0ff8a-c8bf-43ff-9b4e-3d8050932fd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:39.925525 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde0ff8a-c8bf-43ff-9b4e-3d8050932fd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:39.925525 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde0ff8a-c8bf-43ff-9b4e-3d8050932fd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:39.930497 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde0ff8a-c8bf-43ff-9b4e-3d8050932fd0 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:39.951507 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 863/1711] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:03:39 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 10 13:03:39.960975 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde0ff8a-c8bf-43ff-9b4e-3d8050932fd0 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:39.962687 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:03:39.991512 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde0ff8a-c8bf-43ff-9b4e-3d8050932fd0 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:40.005246 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2d1fbcd3-d30b-4f95-9138-b6cda99df77d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e51c012a9b3438fa8cf3af68950bdec', 'request_id': 'req-2d1fbcd3-d30b-4f95-9138-b6cda99df77d'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:03:40.025684 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2d1fbcd3-d30b-4f95-9138-b6cda99df77d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e51c012a9b3438fa8cf3af68950bdec', 'request_id': 'req-2d1fbcd3-d30b-4f95-9138-b6cda99df77d'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:03:40.058792 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2d1fbcd3-d30b-4f95-9138-b6cda99df77d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e51c012a9b3438fa8cf3af68950bdec', 'request_id': 'req-2d1fbcd3-d30b-4f95-9138-b6cda99df77d'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:03:40.140964 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:03:40.143439 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2d1fbcd3-d30b-4f95-9138-b6cda99df77d admin admin] The token cache is being invalidate because project 4e51c012a9b3438fa8cf3af68950bdec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:03:40.143734 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2d1fbcd3-d30b-4f95-9138-b6cda99df77d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:03:40.144485 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2d1fbcd3-d30b-4f95-9138-b6cda99df77d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:03:40.146366 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 849/1712] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:03:39 2022] DELETE /identity/v3/projects/4e51c012a9b3438fa8cf3af68950bdec => generated 0 bytes in 752 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:03:40.151898 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1fbcd3-d30b-4f95-9138-b6cda99df77d admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:40.153658 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-2d1fbcd3-d30b-4f95-9138-b6cda99df77d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:40.157219 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9796bc4-14ec-418d-ab69-85238655a65d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:40.160810 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9796bc4-14ec-418d-ab69-85238655a65d admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:40.160810 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9796bc4-14ec-418d-ab69-85238655a65d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:40.160810 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9796bc4-14ec-418d-ab69-85238655a65d admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:40.161120 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9796bc4-14ec-418d-ab69-85238655a65d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:40.164871 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9796bc4-14ec-418d-ab69-85238655a65d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:40.179778 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1fbcd3-d30b-4f95-9138-b6cda99df77d admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:40.193964 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 864/1713] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:03:39 2022] GET /identity/v3/roles => generated 1420 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:03:40.197632 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9796bc4-14ec-418d-ab69-85238655a65d admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:40.200619 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f9796bc4-14ec-418d-ab69-85238655a65d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:40.210867 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4ebccb-cfa3-4f34-8b00-fe0f5f9dfcb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:40.210867 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4ebccb-cfa3-4f34-8b00-fe0f5f9dfcb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:40.210867 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4ebccb-cfa3-4f34-8b00-fe0f5f9dfcb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:40.210867 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4ebccb-cfa3-4f34-8b00-fe0f5f9dfcb1 admin admin] PATH_INFO: `/v3/users/a6786c83c72d410f99ad5521b523cb26` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:40.214931 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4ebccb-cfa3-4f34-8b00-fe0f5f9dfcb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6786c83c72d410f99ad5521b523cb26)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:40.220786 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4ebccb-cfa3-4f34-8b00-fe0f5f9dfcb1 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:40.231967 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9796bc4-14ec-418d-ab69-85238655a65d admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:40.249886 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fce610-99c4-4b0e-a115-4331ccad9f5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:40.250999 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fce610-99c4-4b0e-a115-4331ccad9f5b admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:40.253060 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fce610-99c4-4b0e-a115-4331ccad9f5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:40.253388 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fce610-99c4-4b0e-a115-4331ccad9f5b admin admin] PATH_INFO: `/v3/projects/154b5a97925f4504b48077949b6bf7df/users/1e69e929a7a44c1f972874a5ed8f5945/roles/43106c664e1b40de984c0c62701149c8` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:40.297002 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fce610-99c4-4b0e-a115-4331ccad9f5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=154b5a97925f4504b48077949b6bf7df, user_id=1e69e929a7a44c1f972874a5ed8f5945, role_id=43106c664e1b40de984c0c62701149c8)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:40.304780 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fce610-99c4-4b0e-a115-4331ccad9f5b admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:40.419531 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 865/1714] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:03:40 2022] PUT /identity/v3/projects/154b5a97925f4504b48077949b6bf7df/users/1e69e929a7a44c1f972874a5ed8f5945/roles/43106c664e1b40de984c0c62701149c8 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:03:40.419531 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b53a6fa-98fe-4695-bd9a-4de22f89dcbe None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:40.419531 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b53a6fa-98fe-4695-bd9a-4de22f89dcbe None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:40.419531 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b53a6fa-98fe-4695-bd9a-4de22f89dcbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:40.474177 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ae4ebccb-cfa3-4f34-8b00-fe0f5f9dfcb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6786c83c72d410f99ad5521b523cb26'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:03:40.479444 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-7b53a6fa-98fe-4695-bd9a-4de22f89dcbe None None] MFA Rules not processed for user `8e32dbeb85bd4fb889d6dde932c60853`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:03:40.479444 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-7b53a6fa-98fe-4695-bd9a-4de22f89dcbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:40.505310 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ae4ebccb-cfa3-4f34-8b00-fe0f5f9dfcb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6786c83c72d410f99ad5521b523cb26'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:03:40.589687 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ae4ebccb-cfa3-4f34-8b00-fe0f5f9dfcb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6786c83c72d410f99ad5521b523cb26'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:03:40.590957 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ae4ebccb-cfa3-4f34-8b00-fe0f5f9dfcb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6786c83c72d410f99ad5521b523cb26'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:03:40.624133 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ae4ebccb-cfa3-4f34-8b00-fe0f5f9dfcb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6786c83c72d410f99ad5521b523cb26'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:03:40.631438 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 850/1715] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:03:40 2022] DELETE /identity/v3/users/a6786c83c72d410f99ad5521b523cb26 => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:03:40.632947 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8561b73-f8a6-4ed3-9a17-d5a8978e3bdb None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:40.633465 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8561b73-f8a6-4ed3-9a17-d5a8978e3bdb None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:40.633996 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8561b73-f8a6-4ed3-9a17-d5a8978e3bdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:40.666065 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 866/1716] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:03:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:03:40.675097 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-a8561b73-f8a6-4ed3-9a17-d5a8978e3bdb None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:03:40.676363 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a8561b73-f8a6-4ed3-9a17-d5a8978e3bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:40.681571 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ca8037-e5f9-42a1-9e21-4ade16f178d5 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:40.681990 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ca8037-e5f9-42a1-9e21-4ade16f178d5 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:40.682268 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ca8037-e5f9-42a1-9e21-4ade16f178d5 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:40.686287 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 867/1717] 10.222.0.27 () {54 vars in 921 bytes} [Fri Jun 10 13:03:40 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 13:03:40.687799 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ca8037-e5f9-42a1-9e21-4ade16f178d5 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:40.690593 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b4ca8037-e5f9-42a1-9e21-4ade16f178d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:40.841963 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 851/1718] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:03:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:03:40.845039 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40a5626-0c6e-49a5-88aa-8984e732c6c3 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:40.845554 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40a5626-0c6e-49a5-88aa-8984e732c6c3 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:40.849780 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40a5626-0c6e-49a5-88aa-8984e732c6c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:40.924948 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-b40a5626-0c6e-49a5-88aa-8984e732c6c3 None None] MFA Rules not processed for user `8bd6caa461e84100a3d5fe5af5d80f3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:03:40.928662 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b40a5626-0c6e-49a5-88aa-8984e732c6c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:40.965672 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ca8037-e5f9-42a1-9e21-4ade16f178d5 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:40.970080 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 852/1719] 10.222.0.27 () {58 vars in 1018 bytes} [Fri Jun 10 13:03:40 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:03:40.973032 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40a5626-0c6e-49a5-88aa-8984e732c6c3 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:40.975108 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14cbb53-6ab8-4ad6-afc3-73079a3952f1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bd6caa461e84100a3d5fe5af5d80f3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:40.976131 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14cbb53-6ab8-4ad6-afc3-73079a3952f1 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:40.977041 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14cbb53-6ab8-4ad6-afc3-73079a3952f1 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:40.978111 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b40a5626-0c6e-49a5-88aa-8984e732c6c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:40.980702 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14cbb53-6ab8-4ad6-afc3-73079a3952f1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:41.116678 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:03:41.226192 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:03:41.247115 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40a5626-0c6e-49a5-88aa-8984e732c6c3 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:41.249789 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14cbb53-6ab8-4ad6-afc3-73079a3952f1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:41.253718 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14cbb53-6ab8-4ad6-afc3-73079a3952f1 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:41.262694 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d95887-1c44-48cf-9dbd-beec41eeb92e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:41.263383 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d95887-1c44-48cf-9dbd-beec41eeb92e service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:41.263674 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d95887-1c44-48cf-9dbd-beec41eeb92e service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:41.269175 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d95887-1c44-48cf-9dbd-beec41eeb92e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:41.312599 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 868/1720] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:03:40 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 624 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:03:41.326862 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d95887-1c44-48cf-9dbd-beec41eeb92e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:41.329188 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d95887-1c44-48cf-9dbd-beec41eeb92e service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:41.352676 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 853/1721] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:03:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:03:41.621444 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14cbb53-6ab8-4ad6-afc3-73079a3952f1 service nova] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:41.640929 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14cbb53-6ab8-4ad6-afc3-73079a3952f1 service nova] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:41.658521 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e22ed3-52a5-406f-b121-6441a6f5b2f6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bd6caa461e84100a3d5fe5af5d80f3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:41.659344 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e22ed3-52a5-406f-b121-6441a6f5b2f6 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:41.659830 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e22ed3-52a5-406f-b121-6441a6f5b2f6 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:41.664071 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e22ed3-52a5-406f-b121-6441a6f5b2f6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:41.686572 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e22ed3-52a5-406f-b121-6441a6f5b2f6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:41.688343 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e22ed3-52a5-406f-b121-6441a6f5b2f6 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:41.711586 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 869/1722] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:03:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:03:42.493670 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f15571-ec04-4dc6-b129-97dc18a7fe31 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:42.494250 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f15571-ec04-4dc6-b129-97dc18a7fe31 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:42.494250 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f15571-ec04-4dc6-b129-97dc18a7fe31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:42.617331 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-03f15571-ec04-4dc6-b129-97dc18a7fe31 None None] MFA Rules not processed for user `e0b63da5bf0b4410acc9ff573064f8ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:03:42.618851 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-03f15571-ec04-4dc6-b129-97dc18a7fe31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:42.685223 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e22ed3-52a5-406f-b121-6441a6f5b2f6 service nova] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:42.687200 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c2e22ed3-52a5-406f-b121-6441a6f5b2f6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:42.749431 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e22ed3-52a5-406f-b121-6441a6f5b2f6 service nova] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:42.784599 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57905586-8404-4608-a052-c1d5d43ba6d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:42.785477 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57905586-8404-4608-a052-c1d5d43ba6d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:42.785756 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57905586-8404-4608-a052-c1d5d43ba6d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:42.786036 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57905586-8404-4608-a052-c1d5d43ba6d7 admin admin] PATH_INFO: `/v3/projects/a3334230920d4b8c8b098837ea9bb4b6` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:42.790767 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57905586-8404-4608-a052-c1d5d43ba6d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3334230920d4b8c8b098837ea9bb4b6)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:42.794521 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57905586-8404-4608-a052-c1d5d43ba6d7 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:42.984196 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 854/1723] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:03:42 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 492 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:03:42.996175 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103a9597-f6fd-489c-95d6-6c2c5401ab1c None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:42.998944 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103a9597-f6fd-489c-95d6-6c2c5401ab1c None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:42.999587 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103a9597-f6fd-489c-95d6-6c2c5401ab1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:43.087191 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-103a9597-f6fd-489c-95d6-6c2c5401ab1c None None] MFA Rules not processed for user `e0b63da5bf0b4410acc9ff573064f8ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:03:43.089911 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-103a9597-f6fd-489c-95d6-6c2c5401ab1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:43.117867 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-57905586-8404-4608-a052-c1d5d43ba6d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3334230920d4b8c8b098837ea9bb4b6', 'request_id': 'req-57905586-8404-4608-a052-c1d5d43ba6d7'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:03:43.139686 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-57905586-8404-4608-a052-c1d5d43ba6d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3334230920d4b8c8b098837ea9bb4b6', 'request_id': 'req-57905586-8404-4608-a052-c1d5d43ba6d7'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:03:43.160891 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-57905586-8404-4608-a052-c1d5d43ba6d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3334230920d4b8c8b098837ea9bb4b6', 'request_id': 'req-57905586-8404-4608-a052-c1d5d43ba6d7'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:03:43.241270 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-57905586-8404-4608-a052-c1d5d43ba6d7 admin admin] The token cache is being invalidate because project a3334230920d4b8c8b098837ea9bb4b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:03:43.241561 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-57905586-8404-4608-a052-c1d5d43ba6d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:03:43.242509 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-57905586-8404-4608-a052-c1d5d43ba6d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:03:43.243691 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 855/1724] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:03:42 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:03:43.251281 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103a9597-f6fd-489c-95d6-6c2c5401ab1c None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:43.252917 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-103a9597-f6fd-489c-95d6-6c2c5401ab1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:43.255401 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 870/1725] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:03:42 2022] DELETE /identity/v3/projects/a3334230920d4b8c8b098837ea9bb4b6 => generated 0 bytes in 571 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 10 13:03:43.260806 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b350a8-597d-47e2-97ba-27c471ee0ec4 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:43.267066 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b350a8-597d-47e2-97ba-27c471ee0ec4 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:43.267646 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b350a8-597d-47e2-97ba-27c471ee0ec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:43.314523 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-31b350a8-597d-47e2-97ba-27c471ee0ec4 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:03:43.316434 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-31b350a8-597d-47e2-97ba-27c471ee0ec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:43.458218 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 871/1726] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:03:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:03:43.465448 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd51b643-7788-40be-b4d9-82257be7f46f None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:43.465938 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd51b643-7788-40be-b4d9-82257be7f46f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:43.466999 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd51b643-7788-40be-b4d9-82257be7f46f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:43.551719 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-bd51b643-7788-40be-b4d9-82257be7f46f None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:03:43.557669 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-bd51b643-7788-40be-b4d9-82257be7f46f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:43.586357 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:03:43.595826 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 872/1727] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:03:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:03:43.608230 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd51b643-7788-40be-b4d9-82257be7f46f None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:43.617395 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103a9597-f6fd-489c-95d6-6c2c5401ab1c None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:43.635839 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd51b643-7788-40be-b4d9-82257be7f46f None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:43.643168 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6b74a0-71b4-4a39-8cc2-6211a0fcde13 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bd6caa461e84100a3d5fe5af5d80f3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:43.655823 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b337ee3-3d8a-4723-8cbb-44b35b5eacbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:43.659263 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b337ee3-3d8a-4723-8cbb-44b35b5eacbd admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:43.660212 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b337ee3-3d8a-4723-8cbb-44b35b5eacbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:43.660212 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b337ee3-3d8a-4723-8cbb-44b35b5eacbd admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:43.661060 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b337ee3-3d8a-4723-8cbb-44b35b5eacbd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 13:03:43.662974 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b337ee3-3d8a-4723-8cbb-44b35b5eacbd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:43.668723 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b337ee3-3d8a-4723-8cbb-44b35b5eacbd admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:43.674130 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6b74a0-71b4-4a39-8cc2-6211a0fcde13 service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:43.675276 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6b74a0-71b4-4a39-8cc2-6211a0fcde13 service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:43.675785 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6b74a0-71b4-4a39-8cc2-6211a0fcde13 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:43.679952 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-4e6b74a0-71b4-4a39-8cc2-6211a0fcde13 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:43.704686 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 873/1728] 10.222.0.27 () {58 vars in 1152 bytes} [Fri Jun 10 13:03:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 13:03:43.715071 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b337ee3-3d8a-4723-8cbb-44b35b5eacbd admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:43.743615 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6b74a0-71b4-4a39-8cc2-6211a0fcde13 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:43.746361 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6b74a0-71b4-4a39-8cc2-6211a0fcde13 service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:43.766260 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b337ee3-3d8a-4723-8cbb-44b35b5eacbd admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:43.783812 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84892736-7bcd-469f-8b21-07cf4071cf8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:43.784861 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84892736-7bcd-469f-8b21-07cf4071cf8c admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:43.785352 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84892736-7bcd-469f-8b21-07cf4071cf8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:43.785809 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84892736-7bcd-469f-8b21-07cf4071cf8c admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:43.787250 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84892736-7bcd-469f-8b21-07cf4071cf8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:43.789788 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84892736-7bcd-469f-8b21-07cf4071cf8c admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:43.789902 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 856/1729] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:03:43 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 539 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:03:43.957516 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 874/1730] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:03:43 2022] POST /identity/v3/projects => generated 375 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:03:43.962958 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6b74a0-71b4-4a39-8cc2-6211a0fcde13 service nova] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:44.001197 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6b74a0-71b4-4a39-8cc2-6211a0fcde13 service nova] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:44.132581 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0f9c50-b317-460d-8cf9-7ec40f0bc7d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:44.133257 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0f9c50-b317-460d-8cf9-7ec40f0bc7d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:44.133570 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0f9c50-b317-460d-8cf9-7ec40f0bc7d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:44.133891 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0f9c50-b317-460d-8cf9-7ec40f0bc7d2 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:44.134591 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0f9c50-b317-460d-8cf9-7ec40f0bc7d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:44.137242 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0f9c50-b317-460d-8cf9-7ec40f0bc7d2 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:44.266006 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 857/1731] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:03:43 2022] POST /identity/v3/users => generated 288 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:03:44.267817 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84892736-7bcd-469f-8b21-07cf4071cf8c admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:44.286390 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84892736-7bcd-469f-8b21-07cf4071cf8c admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:44.298547 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0589a26-bd0a-4427-927d-45009efba1fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:44.300621 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0589a26-bd0a-4427-927d-45009efba1fb admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:44.301942 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0589a26-bd0a-4427-927d-45009efba1fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:44.303136 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0589a26-bd0a-4427-927d-45009efba1fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:44.306621 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0589a26-bd0a-4427-927d-45009efba1fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:44.309845 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0589a26-bd0a-4427-927d-45009efba1fb admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:44.350249 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0f9c50-b317-460d-8cf9-7ec40f0bc7d2 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:44.352471 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 875/1732] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:03:44 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 10 13:03:44.371775 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0f9c50-b317-460d-8cf9-7ec40f0bc7d2 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:44.393043 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0a6d2f-aef2-4c50-b1d5-822597ab4520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:44.394330 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0a6d2f-aef2-4c50-b1d5-822597ab4520 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:44.395806 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0a6d2f-aef2-4c50-b1d5-822597ab4520 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:44.396491 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0a6d2f-aef2-4c50-b1d5-822597ab4520 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:44.400743 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0a6d2f-aef2-4c50-b1d5-822597ab4520 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:44.406949 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0a6d2f-aef2-4c50-b1d5-822597ab4520 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:44.441524 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0589a26-bd0a-4427-927d-45009efba1fb admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:44.442122 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 858/1733] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:03:44 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:03:44.458348 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0589a26-bd0a-4427-927d-45009efba1fb admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:44.469321 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bbf4ba-c16b-41fe-abb5-4c347bf57cd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:44.477098 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bbf4ba-c16b-41fe-abb5-4c347bf57cd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:44.477098 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bbf4ba-c16b-41fe-abb5-4c347bf57cd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:44.477098 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bbf4ba-c16b-41fe-abb5-4c347bf57cd9 admin admin] PATH_INFO: `/v3/projects/911ab827aa504e5596c79ba278d157a8/users/05108a8d34ca4060b5a4c18b24556fba/roles/43106c664e1b40de984c0c62701149c8` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:44.503799 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:03:44.520537 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bbf4ba-c16b-41fe-abb5-4c347bf57cd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=911ab827aa504e5596c79ba278d157a8, user_id=05108a8d34ca4060b5a4c18b24556fba, role_id=43106c664e1b40de984c0c62701149c8)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:44.524173 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bbf4ba-c16b-41fe-abb5-4c347bf57cd9 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:44.640155 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 876/1734] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:03:44 2022] PUT /identity/v3/projects/911ab827aa504e5596c79ba278d157a8/users/05108a8d34ca4060b5a4c18b24556fba/roles/43106c664e1b40de984c0c62701149c8 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:03:44.650188 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49094987-dc0c-496e-945e-6a99c32ca7b2 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:44.650762 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49094987-dc0c-496e-945e-6a99c32ca7b2 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:44.650870 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49094987-dc0c-496e-945e-6a99c32ca7b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:44.704373 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-49094987-dc0c-496e-945e-6a99c32ca7b2 None None] MFA Rules not processed for user `05108a8d34ca4060b5a4c18b24556fba`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:03:44.707388 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-49094987-dc0c-496e-945e-6a99c32ca7b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:44.952963 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 859/1735] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:03:44 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:03:44.953338 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bbf4ba-c16b-41fe-abb5-4c347bf57cd9 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:45.002986 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e5c81f-5645-41ce-8ae8-d01a022bc404 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:45.003965 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e5c81f-5645-41ce-8ae8-d01a022bc404 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:45.003965 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e5c81f-5645-41ce-8ae8-d01a022bc404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:45.039679 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-f5e5c81f-5645-41ce-8ae8-d01a022bc404 None None] MFA Rules not processed for user `634e6116f688494c9e11d59479f94a43`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:03:45.040934 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f5e5c81f-5645-41ce-8ae8-d01a022bc404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:45.207054 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 860/1736] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:03:45 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:03:45.272353 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bbf4ba-c16b-41fe-abb5-4c347bf57cd9 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:45.292616 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3402354-fae8-4de7-a6c2-fee696737fbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:45.293661 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3402354-fae8-4de7-a6c2-fee696737fbd admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:45.294418 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3402354-fae8-4de7-a6c2-fee696737fbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:45.294915 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3402354-fae8-4de7-a6c2-fee696737fbd admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:45.295814 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3402354-fae8-4de7-a6c2-fee696737fbd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:45.297972 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3402354-fae8-4de7-a6c2-fee696737fbd admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:45.447787 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 877/1737] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:03:44 2022] POST /identity/v3/projects => generated 377 bytes in 495 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 10 13:03:45.452486 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e5c81f-5645-41ce-8ae8-d01a022bc404 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:45.458684 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:03:45.469492 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e5c81f-5645-41ce-8ae8-d01a022bc404 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:45.646581 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e898cb-26a1-480e-b96c-2709984a48de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:45.647342 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e898cb-26a1-480e-b96c-2709984a48de admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:45.664772 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e898cb-26a1-480e-b96c-2709984a48de admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:45.665431 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e898cb-26a1-480e-b96c-2709984a48de admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:45.665824 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e898cb-26a1-480e-b96c-2709984a48de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:45.669122 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e898cb-26a1-480e-b96c-2709984a48de admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:45.825669 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 861/1738] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:03:45 2022] POST /identity/v3/users => generated 295 bytes in 374 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:03:45.832502 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3402354-fae8-4de7-a6c2-fee696737fbd admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:45.850920 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3402354-fae8-4de7-a6c2-fee696737fbd admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:45.861826 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d13a9e3-5f3d-4c71-ab68-3070c99e4c2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:45.862906 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d13a9e3-5f3d-4c71-ab68-3070c99e4c2e admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:45.863394 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d13a9e3-5f3d-4c71-ab68-3070c99e4c2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:45.863890 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d13a9e3-5f3d-4c71-ab68-3070c99e4c2e admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:45.864847 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d13a9e3-5f3d-4c71-ab68-3070c99e4c2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:45.866830 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d13a9e3-5f3d-4c71-ab68-3070c99e4c2e admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:45.892353 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 878/1739] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:03:45 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:03:45.898067 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e898cb-26a1-480e-b96c-2709984a48de admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:45.922937 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e898cb-26a1-480e-b96c-2709984a48de admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:45.930017 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77125be8-5153-467c-9b2a-384142e0f332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:45.930993 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77125be8-5153-467c-9b2a-384142e0f332 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:45.937392 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77125be8-5153-467c-9b2a-384142e0f332 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:45.937721 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77125be8-5153-467c-9b2a-384142e0f332 admin admin] PATH_INFO: `/v3/projects/f20717032f92451282c865fb118ab429/users/c2d90f26d2c348dd86c9be001a3b8603/roles/b43d5a8247e244869c2c70f986f42e8a` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:45.999795 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77125be8-5153-467c-9b2a-384142e0f332 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f20717032f92451282c865fb118ab429, user_id=c2d90f26d2c348dd86c9be001a3b8603, role_id=b43d5a8247e244869c2c70f986f42e8a)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:46.014455 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77125be8-5153-467c-9b2a-384142e0f332 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:46.161538 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 862/1740] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:03:45 2022] PUT /identity/v3/projects/f20717032f92451282c865fb118ab429/users/c2d90f26d2c348dd86c9be001a3b8603/roles/b43d5a8247e244869c2c70f986f42e8a => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 10 13:03:46.165562 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d13a9e3-5f3d-4c71-ab68-3070c99e4c2e admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:46.181629 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d13a9e3-5f3d-4c71-ab68-3070c99e4c2e admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:46.354477 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072b6ce5-d116-4d57-aff2-09dbe1312fda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:46.355184 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072b6ce5-d116-4d57-aff2-09dbe1312fda admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:46.355472 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072b6ce5-d116-4d57-aff2-09dbe1312fda admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:46.356663 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072b6ce5-d116-4d57-aff2-09dbe1312fda admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:46.356663 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072b6ce5-d116-4d57-aff2-09dbe1312fda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:46.358234 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072b6ce5-d116-4d57-aff2-09dbe1312fda admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:46.421327 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77125be8-5153-467c-9b2a-384142e0f332 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:46.424014 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 879/1741] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:03:46 2022] GET /identity/v3/roles => generated 1420 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:03:46.438078 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77125be8-5153-467c-9b2a-384142e0f332 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:46.482620 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4317d9f-c24a-4c0c-9d05-54941acd50b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:46.483545 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4317d9f-c24a-4c0c-9d05-54941acd50b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:46.483831 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4317d9f-c24a-4c0c-9d05-54941acd50b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:46.484141 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4317d9f-c24a-4c0c-9d05-54941acd50b2 admin admin] PATH_INFO: `/v3/projects/f20717032f92451282c865fb118ab429/users/c2d90f26d2c348dd86c9be001a3b8603/roles/b43d5a8247e244869c2c70f986f42e8a` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:46.502274 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4317d9f-c24a-4c0c-9d05-54941acd50b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f20717032f92451282c865fb118ab429, user_id=c2d90f26d2c348dd86c9be001a3b8603, role_id=b43d5a8247e244869c2c70f986f42e8a)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:46.504987 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4317d9f-c24a-4c0c-9d05-54941acd50b2 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:46.568861 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 863/1742] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:03:46 2022] PUT /identity/v3/projects/f20717032f92451282c865fb118ab429/users/c2d90f26d2c348dd86c9be001a3b8603/roles/b43d5a8247e244869c2c70f986f42e8a => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:03:46.571422 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac65cff4-c925-4b6d-ba9a-04c014f8aa89 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:46.572011 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac65cff4-c925-4b6d-ba9a-04c014f8aa89 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:46.572510 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac65cff4-c925-4b6d-ba9a-04c014f8aa89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:46.652695 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-ac65cff4-c925-4b6d-ba9a-04c014f8aa89 None None] MFA Rules not processed for user `c2d90f26d2c348dd86c9be001a3b8603`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:03:46.655768 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ac65cff4-c925-4b6d-ba9a-04c014f8aa89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:46.661865 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befcbf47-f723-469c-ac80-427f38dcb360 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:46.662328 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befcbf47-f723-469c-ac80-427f38dcb360 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:46.662749 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befcbf47-f723-469c-ac80-427f38dcb360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:46.686567 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-befcbf47-f723-469c-ac80-427f38dcb360 None None] MFA Rules not processed for user `e0b63da5bf0b4410acc9ff573064f8ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:03:46.688466 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-befcbf47-f723-469c-ac80-427f38dcb360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:46.805029 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:03:46.845636 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 864/1743] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:03:46 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:03:46.847290 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b49201-8d91-4a81-a0f4-8c4965fd0196 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:46.847762 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b49201-8d91-4a81-a0f4-8c4965fd0196 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:46.848522 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b49201-8d91-4a81-a0f4-8c4965fd0196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:46.904114 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 880/1744] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:03:46 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:03:46.912669 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac65cff4-c925-4b6d-ba9a-04c014f8aa89 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:46.914039 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-d6b49201-8d91-4a81-a0f4-8c4965fd0196 None None] MFA Rules not processed for user `c71b903e00214894b94b03b3401607a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:03:46.914700 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ac65cff4-c925-4b6d-ba9a-04c014f8aa89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:46.915729 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d6b49201-8d91-4a81-a0f4-8c4965fd0196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:47.088006 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 865/1745] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:03:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:03:47.112270 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac65cff4-c925-4b6d-ba9a-04c014f8aa89 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:47.127241 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174ca0da-5237-453d-af89-839c6bd2ea19 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6cc95a563b941ac92ab283aa0d5a5d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:47.128200 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174ca0da-5237-453d-af89-839c6bd2ea19 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:47.128562 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174ca0da-5237-453d-af89-839c6bd2ea19 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:47.128914 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174ca0da-5237-453d-af89-839c6bd2ea19 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:47.276885 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174ca0da-5237-453d-af89-839c6bd2ea19 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:47.279188 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174ca0da-5237-453d-af89-839c6bd2ea19 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:47.311408 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 881/1746] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:03:46 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 10 13:03:47.519533 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18790711-1d64-4216-8124-3e5891ac6105 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:47.520124 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18790711-1d64-4216-8124-3e5891ac6105 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:47.520569 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18790711-1d64-4216-8124-3e5891ac6105 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:47.522550 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 866/1747] 10.222.0.27 () {54 vars in 912 bytes} [Fri Jun 10 13:03:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 13:03:47.535971 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea28a93-8e9a-403d-bff3-0e3815f1e3f0 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:47.536743 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea28a93-8e9a-403d-bff3-0e3815f1e3f0 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:47.537078 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea28a93-8e9a-403d-bff3-0e3815f1e3f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:47.593694 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-6ea28a93-8e9a-403d-bff3-0e3815f1e3f0 None None] MFA Rules not processed for user `bc4415461a974c24ad7105a97c7d915b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:03:47.603695 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6ea28a93-8e9a-403d-bff3-0e3815f1e3f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:47.768363 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 882/1748] 10.222.0.27 () {58 vars in 1009 bytes} [Fri Jun 10 13:03:47 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 10 13:03:47.783621 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18790711-1d64-4216-8124-3e5891ac6105 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:47.787247 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-18790711-1d64-4216-8124-3e5891ac6105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:47.802163 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4e3399-11a9-4db7-ace9-da76b3bdd7cb None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:47.804139 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4e3399-11a9-4db7-ace9-da76b3bdd7cb None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:47.805204 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4e3399-11a9-4db7-ace9-da76b3bdd7cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:47.815296 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:03:47.827636 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18790711-1d64-4216-8124-3e5891ac6105 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:47.836705 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9851561-bc17-4d09-8344-c4d6882cdd01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:47.839842 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9851561-bc17-4d09-8344-c4d6882cdd01 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:47.839842 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9851561-bc17-4d09-8344-c4d6882cdd01 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:47.839842 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9851561-bc17-4d09-8344-c4d6882cdd01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:47.854177 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9851561-bc17-4d09-8344-c4d6882cdd01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:47.855917 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9851561-bc17-4d09-8344-c4d6882cdd01 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:47.873295 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-6a4e3399-11a9-4db7-ace9-da76b3bdd7cb None None] MFA Rules not processed for user `1e69e929a7a44c1f972874a5ed8f5945`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:03:47.874958 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6a4e3399-11a9-4db7-ace9-da76b3bdd7cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:47.885071 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 867/1749] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:03:47 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:03:47.965611 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:03:48.046409 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9851561-bc17-4d09-8344-c4d6882cdd01 service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:48.048255 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e9851561-bc17-4d09-8344-c4d6882cdd01 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:48.049492 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 883/1750] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:03:47 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:03:48.185215 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9851561-bc17-4d09-8344-c4d6882cdd01 service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:48.203894 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd469eb3-88c4-4d76-97f3-149ff4881be1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:48.205818 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd469eb3-88c4-4d76-97f3-149ff4881be1 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:48.210627 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd469eb3-88c4-4d76-97f3-149ff4881be1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:48.211326 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd469eb3-88c4-4d76-97f3-149ff4881be1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:48.212073 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd469eb3-88c4-4d76-97f3-149ff4881be1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:48.217861 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd469eb3-88c4-4d76-97f3-149ff4881be1 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:48.390057 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 868/1751] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:03:48 2022] POST /identity/v3/projects => generated 405 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:03:48.394382 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4e3399-11a9-4db7-ace9-da76b3bdd7cb None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:48.413088 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4e3399-11a9-4db7-ace9-da76b3bdd7cb None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:48.567217 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010a1c7e-cf75-44c9-8c0e-1d846773899a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:48.567946 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010a1c7e-cf75-44c9-8c0e-1d846773899a admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:48.568241 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010a1c7e-cf75-44c9-8c0e-1d846773899a admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:48.571546 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010a1c7e-cf75-44c9-8c0e-1d846773899a admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:48.572345 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010a1c7e-cf75-44c9-8c0e-1d846773899a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:48.575460 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010a1c7e-cf75-44c9-8c0e-1d846773899a admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:48.782325 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 884/1752] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:03:48 2022] POST /identity/v3/users => generated 309 bytes in 386 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:03:48.784301 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd469eb3-88c4-4d76-97f3-149ff4881be1 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:48.799690 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd469eb3-88c4-4d76-97f3-149ff4881be1 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:48.834383 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cc4a33-c1b3-4385-b6d7-d66392226f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:48.835336 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cc4a33-c1b3-4385-b6d7-d66392226f41 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:48.835841 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cc4a33-c1b3-4385-b6d7-d66392226f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:48.836472 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cc4a33-c1b3-4385-b6d7-d66392226f41 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:48.837427 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cc4a33-c1b3-4385-b6d7-d66392226f41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:48.841219 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cc4a33-c1b3-4385-b6d7-d66392226f41 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:48.863700 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 869/1753] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:03:48 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:03:48.868230 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010a1c7e-cf75-44c9-8c0e-1d846773899a admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:48.888874 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010a1c7e-cf75-44c9-8c0e-1d846773899a admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:48.929914 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d4c03f-749b-4df7-ba75-cde0fff9f52b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:48.930846 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d4c03f-749b-4df7-ba75-cde0fff9f52b admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:48.931132 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d4c03f-749b-4df7-ba75-cde0fff9f52b admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:48.931413 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d4c03f-749b-4df7-ba75-cde0fff9f52b admin admin] PATH_INFO: `/v3/projects/8eb04aedc9b84408ad7405342b0a92ad/users/fd2a1f3ec0af4da9bd491d669de97a27/roles/b43d5a8247e244869c2c70f986f42e8a` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:49.041813 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d4c03f-749b-4df7-ba75-cde0fff9f52b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8eb04aedc9b84408ad7405342b0a92ad, user_id=fd2a1f3ec0af4da9bd491d669de97a27, role_id=b43d5a8247e244869c2c70f986f42e8a)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:49.046675 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d4c03f-749b-4df7-ba75-cde0fff9f52b admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:49.215389 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 885/1754] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:03:48 2022] PUT /identity/v3/projects/8eb04aedc9b84408ad7405342b0a92ad/users/fd2a1f3ec0af4da9bd491d669de97a27/roles/b43d5a8247e244869c2c70f986f42e8a => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 10 13:03:49.218672 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cc4a33-c1b3-4385-b6d7-d66392226f41 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:49.243363 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cc4a33-c1b3-4385-b6d7-d66392226f41 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:49.352076 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546ae4cf-b0ad-4428-982d-f95c645f41c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:49.357602 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546ae4cf-b0ad-4428-982d-f95c645f41c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:49.357602 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546ae4cf-b0ad-4428-982d-f95c645f41c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:49.357602 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546ae4cf-b0ad-4428-982d-f95c645f41c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:49.357602 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546ae4cf-b0ad-4428-982d-f95c645f41c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:49.359721 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546ae4cf-b0ad-4428-982d-f95c645f41c2 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:49.381517 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 870/1755] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:03:49 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:03:49.383581 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d4c03f-749b-4df7-ba75-cde0fff9f52b admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:49.404975 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d4c03f-749b-4df7-ba75-cde0fff9f52b admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:49.421771 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976349ae-caa2-4bbf-9167-5647d88455b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:49.423068 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976349ae-caa2-4bbf-9167-5647d88455b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:49.424595 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976349ae-caa2-4bbf-9167-5647d88455b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:49.425985 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976349ae-caa2-4bbf-9167-5647d88455b6 admin admin] PATH_INFO: `/v3/projects/8eb04aedc9b84408ad7405342b0a92ad/users/fd2a1f3ec0af4da9bd491d669de97a27/roles/b43d5a8247e244869c2c70f986f42e8a` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:49.434768 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976349ae-caa2-4bbf-9167-5647d88455b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8eb04aedc9b84408ad7405342b0a92ad, user_id=fd2a1f3ec0af4da9bd491d669de97a27, role_id=b43d5a8247e244869c2c70f986f42e8a)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:49.444501 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976349ae-caa2-4bbf-9167-5647d88455b6 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:49.465114 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 886/1756] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:03:49 2022] PUT /identity/v3/projects/8eb04aedc9b84408ad7405342b0a92ad/users/fd2a1f3ec0af4da9bd491d669de97a27/roles/b43d5a8247e244869c2c70f986f42e8a => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:03:50.150178 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8c9b5f-2b5b-423e-97a1-ade46df1a65d None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:50.150643 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8c9b5f-2b5b-423e-97a1-ade46df1a65d None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:50.150739 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8c9b5f-2b5b-423e-97a1-ade46df1a65d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:50.170024 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-bb8c9b5f-2b5b-423e-97a1-ade46df1a65d None None] MFA Rules not processed for user `8e32dbeb85bd4fb889d6dde932c60853`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:03:50.171559 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-bb8c9b5f-2b5b-423e-97a1-ade46df1a65d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:50.287932 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 871/1757] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:03:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:03:52.035293 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdcdde1-c1d7-4f9c-b7ba-300e9f1be80d None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:52.035939 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdcdde1-c1d7-4f9c-b7ba-300e9f1be80d None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:52.035939 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdcdde1-c1d7-4f9c-b7ba-300e9f1be80d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:52.075772 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-fbdcdde1-c1d7-4f9c-b7ba-300e9f1be80d None None] MFA Rules not processed for user `fd2a1f3ec0af4da9bd491d669de97a27`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:03:52.078367 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-fbdcdde1-c1d7-4f9c-b7ba-300e9f1be80d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:52.221952 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 887/1758] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:03:52 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 10 13:03:52.771690 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8c9b5f-2b5b-423e-97a1-ade46df1a65d None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:52.775087 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-bb8c9b5f-2b5b-423e-97a1-ade46df1a65d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:52.787288 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:03:52.909341 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8c9b5f-2b5b-423e-97a1-ade46df1a65d None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:52.920586 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e140c3-06ef-4445-8164-335de8e520a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6cc95a563b941ac92ab283aa0d5a5d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:52.921412 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e140c3-06ef-4445-8164-335de8e520a4 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:52.921846 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e140c3-06ef-4445-8164-335de8e520a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:52.922225 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e140c3-06ef-4445-8164-335de8e520a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:53.022562 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e140c3-06ef-4445-8164-335de8e520a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:53.024599 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e140c3-06ef-4445-8164-335de8e520a4 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:53.055475 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 872/1759] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:03:52 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:03:54.468541 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392276bd-60e2-40b1-867d-8e9fbba88c1c None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:54.469657 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392276bd-60e2-40b1-867d-8e9fbba88c1c None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:54.470241 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392276bd-60e2-40b1-867d-8e9fbba88c1c None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:54.472977 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 888/1760] 10.222.0.27 () {54 vars in 921 bytes} [Fri Jun 10 13:03:54 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 13:03:54.509915 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82775948-fcdb-45b8-80d9-47b0d73ca89a None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:54.511584 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82775948-fcdb-45b8-80d9-47b0d73ca89a None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:54.512181 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82775948-fcdb-45b8-80d9-47b0d73ca89a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:54.558425 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-82775948-fcdb-45b8-80d9-47b0d73ca89a None None] MFA Rules not processed for user `d972ab782b3f4d2aba575314f16dafc7`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:03:54.560719 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-82775948-fcdb-45b8-80d9-47b0d73ca89a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:54.710870 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 873/1761] 10.222.0.27 () {58 vars in 1018 bytes} [Fri Jun 10 13:03:54 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:03:54.725048 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392276bd-60e2-40b1-867d-8e9fbba88c1c None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:54.733868 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-392276bd-60e2-40b1-867d-8e9fbba88c1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:54.764054 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392276bd-60e2-40b1-867d-8e9fbba88c1c None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:54.771591 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b144c64f-f6fe-457c-a84a-fdcef08a1178 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd972ab782b3f4d2aba575314f16dafc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:54.772609 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b144c64f-f6fe-457c-a84a-fdcef08a1178 service placement] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:54.773068 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b144c64f-f6fe-457c-a84a-fdcef08a1178 service placement] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:54.773617 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b144c64f-f6fe-457c-a84a-fdcef08a1178 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:54.784622 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b144c64f-f6fe-457c-a84a-fdcef08a1178 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:54.787509 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b144c64f-f6fe-457c-a84a-fdcef08a1178 service placement] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:54.796271 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:03:54.803437 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 889/1762] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:03:54 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:03:56.129797 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349caf06-84ef-4cec-928b-d64d2810486c None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:56.129797 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349caf06-84ef-4cec-928b-d64d2810486c None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:56.129797 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349caf06-84ef-4cec-928b-d64d2810486c None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:56.129797 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 874/1763] 10.222.0.27 () {54 vars in 921 bytes} [Fri Jun 10 13:03:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 13:03:56.152531 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0325d1-52a0-4642-8ee9-ae54e52d980d None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:56.153045 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0325d1-52a0-4642-8ee9-ae54e52d980d None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:56.153119 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0325d1-52a0-4642-8ee9-ae54e52d980d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:56.181407 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-cc0325d1-52a0-4642-8ee9-ae54e52d980d None None] MFA Rules not processed for user `8bd6caa461e84100a3d5fe5af5d80f3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:03:56.184301 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-cc0325d1-52a0-4642-8ee9-ae54e52d980d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:56.308884 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 890/1764] 10.222.0.27 () {58 vars in 1018 bytes} [Fri Jun 10 13:03:56 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:03:56.861796 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349caf06-84ef-4cec-928b-d64d2810486c None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:56.863542 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-349caf06-84ef-4cec-928b-d64d2810486c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:56.887175 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349caf06-84ef-4cec-928b-d64d2810486c None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:56.893273 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196e66c3-fb2c-4427-90d7-c972fd6d90d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bd6caa461e84100a3d5fe5af5d80f3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:56.894189 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196e66c3-fb2c-4427-90d7-c972fd6d90d3 service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:56.894778 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196e66c3-fb2c-4427-90d7-c972fd6d90d3 service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:56.901541 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196e66c3-fb2c-4427-90d7-c972fd6d90d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:56.924484 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196e66c3-fb2c-4427-90d7-c972fd6d90d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:56.927251 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196e66c3-fb2c-4427-90d7-c972fd6d90d3 service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:56.953747 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 875/1765] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:03:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 10 13:03:57.629032 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92735a8b-b490-4eb9-8354-1dafb83dea01 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:57.629590 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92735a8b-b490-4eb9-8354-1dafb83dea01 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:57.629990 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92735a8b-b490-4eb9-8354-1dafb83dea01 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:57.632433 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 891/1766] 10.222.0.27 () {54 vars in 921 bytes} [Fri Jun 10 13:03:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 13:03:57.716102 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefa7a77-1535-476c-ad66-512e86804f7a None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:57.716588 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefa7a77-1535-476c-ad66-512e86804f7a None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:57.716885 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefa7a77-1535-476c-ad66-512e86804f7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:57.770714 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-aefa7a77-1535-476c-ad66-512e86804f7a None None] MFA Rules not processed for user `d972ab782b3f4d2aba575314f16dafc7`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:03:57.772256 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-aefa7a77-1535-476c-ad66-512e86804f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:57.806595 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 876/1767] 10.222.0.27 () {58 vars in 1018 bytes} [Fri Jun 10 13:03:57 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:03:57.858364 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92735a8b-b490-4eb9-8354-1dafb83dea01 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:57.863831 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-92735a8b-b490-4eb9-8354-1dafb83dea01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:57.897506 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92735a8b-b490-4eb9-8354-1dafb83dea01 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:57.907486 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e292ee6a-9f87-4509-843f-ebc06cd80aa5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd972ab782b3f4d2aba575314f16dafc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:57.909335 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e292ee6a-9f87-4509-843f-ebc06cd80aa5 service placement] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:57.910459 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e292ee6a-9f87-4509-843f-ebc06cd80aa5 service placement] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:57.910930 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e292ee6a-9f87-4509-843f-ebc06cd80aa5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:57.927307 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e292ee6a-9f87-4509-843f-ebc06cd80aa5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:57.929366 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e292ee6a-9f87-4509-843f-ebc06cd80aa5 service placement] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:57.941346 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 892/1768] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:03:57 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:03:58.045992 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefa7a77-1535-476c-ad66-512e86804f7a None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:58.099709 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:03:58.150076 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f40275-97ae-455b-bdbc-542cd8397b76 None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:58.150643 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f40275-97ae-455b-bdbc-542cd8397b76 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:58.151064 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f40275-97ae-455b-bdbc-542cd8397b76 None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:58.152950 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 893/1769] 10.222.0.27 () {54 vars in 921 bytes} [Fri Jun 10 13:03:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 13:03:58.160988 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112896e3-f2ee-4644-981d-8024258c4e6b None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:58.161273 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112896e3-f2ee-4644-981d-8024258c4e6b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:58.161696 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112896e3-f2ee-4644-981d-8024258c4e6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:58.193583 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefa7a77-1535-476c-ad66-512e86804f7a None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:58.199812 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-112896e3-f2ee-4644-981d-8024258c4e6b None None] MFA Rules not processed for user `8bd6caa461e84100a3d5fe5af5d80f3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:03:58.202367 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-112896e3-f2ee-4644-981d-8024258c4e6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:58.207425 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8760fcf-effd-43fc-ae08-cd1a1eaa4f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:58.208865 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8760fcf-effd-43fc-ae08-cd1a1eaa4f0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:58.209147 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8760fcf-effd-43fc-ae08-cd1a1eaa4f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:58.210151 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8760fcf-effd-43fc-ae08-cd1a1eaa4f0b admin admin] PATH_INFO: `/v3/users/1e69e929a7a44c1f972874a5ed8f5945` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:58.214780 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8760fcf-effd-43fc-ae08-cd1a1eaa4f0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e69e929a7a44c1f972874a5ed8f5945)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:58.219640 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8760fcf-effd-43fc-ae08-cd1a1eaa4f0b admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:58.226326 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 894/1770] 10.222.0.27 () {58 vars in 1018 bytes} [Fri Jun 10 13:03:58 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 10 13:03:58.384152 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112896e3-f2ee-4644-981d-8024258c4e6b None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:58.398703 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112896e3-f2ee-4644-981d-8024258c4e6b None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:58.409405 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad329eed-6fbc-4d9e-8212-a190aad2c355 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bd6caa461e84100a3d5fe5af5d80f3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:58.410411 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad329eed-6fbc-4d9e-8212-a190aad2c355 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:58.410881 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad329eed-6fbc-4d9e-8212-a190aad2c355 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:58.411647 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad329eed-6fbc-4d9e-8212-a190aad2c355 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:58.425225 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad329eed-6fbc-4d9e-8212-a190aad2c355 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:58.427233 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad329eed-6fbc-4d9e-8212-a190aad2c355 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:58.441213 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 895/1771] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:03:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:03:58.538986 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f8760fcf-effd-43fc-ae08-cd1a1eaa4f0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e69e929a7a44c1f972874a5ed8f5945'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:03:58.549339 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f8760fcf-effd-43fc-ae08-cd1a1eaa4f0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e69e929a7a44c1f972874a5ed8f5945'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:03:58.667174 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f8760fcf-effd-43fc-ae08-cd1a1eaa4f0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e69e929a7a44c1f972874a5ed8f5945'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:03:58.667174 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f8760fcf-effd-43fc-ae08-cd1a1eaa4f0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e69e929a7a44c1f972874a5ed8f5945'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:03:58.680499 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f8760fcf-effd-43fc-ae08-cd1a1eaa4f0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e69e929a7a44c1f972874a5ed8f5945'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:03:58.688543 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 877/1772] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:03:58 2022] DELETE /identity/v3/users/1e69e929a7a44c1f972874a5ed8f5945 => generated 0 bytes in 643 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:03:59.066168 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad329eed-6fbc-4d9e-8212-a190aad2c355 service nova] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:59.102879 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad329eed-6fbc-4d9e-8212-a190aad2c355 service nova] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:59.304260 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ab367c-1bf6-4282-8bbd-338fdb95ddb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:59.305884 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ab367c-1bf6-4282-8bbd-338fdb95ddb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:59.306373 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ab367c-1bf6-4282-8bbd-338fdb95ddb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:59.307452 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ab367c-1bf6-4282-8bbd-338fdb95ddb7 admin admin] PATH_INFO: `/v3/projects/154b5a97925f4504b48077949b6bf7df` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:59.311673 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ab367c-1bf6-4282-8bbd-338fdb95ddb7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=154b5a97925f4504b48077949b6bf7df)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:59.311673 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ab367c-1bf6-4282-8bbd-338fdb95ddb7 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:03:59.504541 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-41ab367c-1bf6-4282-8bbd-338fdb95ddb7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '154b5a97925f4504b48077949b6bf7df', 'request_id': 'req-41ab367c-1bf6-4282-8bbd-338fdb95ddb7'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:03:59.512182 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-41ab367c-1bf6-4282-8bbd-338fdb95ddb7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '154b5a97925f4504b48077949b6bf7df', 'request_id': 'req-41ab367c-1bf6-4282-8bbd-338fdb95ddb7'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:03:59.516480 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:03:59.529293 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-41ab367c-1bf6-4282-8bbd-338fdb95ddb7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '154b5a97925f4504b48077949b6bf7df', 'request_id': 'req-41ab367c-1bf6-4282-8bbd-338fdb95ddb7'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:03:59.658014 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-41ab367c-1bf6-4282-8bbd-338fdb95ddb7 admin admin] The token cache is being invalidate because project 154b5a97925f4504b48077949b6bf7df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:03:59.658401 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-41ab367c-1bf6-4282-8bbd-338fdb95ddb7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:03:59.659278 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-41ab367c-1bf6-4282-8bbd-338fdb95ddb7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:03:59.661100 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 896/1773] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:03:59 2022] DELETE /identity/v3/projects/154b5a97925f4504b48077949b6bf7df => generated 0 bytes in 596 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:03:59.665219 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8760fcf-effd-43fc-ae08-cd1a1eaa4f0b admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:03:59.667952 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f8760fcf-effd-43fc-ae08-cd1a1eaa4f0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:03:59.694997 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8760fcf-effd-43fc-ae08-cd1a1eaa4f0b admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:03:59.781330 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cc6091-d9ac-405c-9043-3efa95ac54a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:03:59.782428 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cc6091-d9ac-405c-9043-3efa95ac54a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:03:59.782713 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cc6091-d9ac-405c-9043-3efa95ac54a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:03:59.783185 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cc6091-d9ac-405c-9043-3efa95ac54a3 admin admin] PATH_INFO: `/v3/users/fd2a1f3ec0af4da9bd491d669de97a27` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:03:59.784663 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cc6091-d9ac-405c-9043-3efa95ac54a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd2a1f3ec0af4da9bd491d669de97a27)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:03:59.786902 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cc6091-d9ac-405c-9043-3efa95ac54a3 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:00.005012 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-a3cc6091-d9ac-405c-9043-3efa95ac54a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd2a1f3ec0af4da9bd491d669de97a27'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:00.014991 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-a3cc6091-d9ac-405c-9043-3efa95ac54a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd2a1f3ec0af4da9bd491d669de97a27'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:00.018196 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:04:00.117935 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-a3cc6091-d9ac-405c-9043-3efa95ac54a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd2a1f3ec0af4da9bd491d669de97a27'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:00.118568 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-a3cc6091-d9ac-405c-9043-3efa95ac54a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd2a1f3ec0af4da9bd491d669de97a27'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:00.136870 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-a3cc6091-d9ac-405c-9043-3efa95ac54a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd2a1f3ec0af4da9bd491d669de97a27'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:00.143464 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 878/1774] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:03:59 2022] DELETE /identity/v3/users/fd2a1f3ec0af4da9bd491d669de97a27 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:04:00.555255 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ab367c-1bf6-4282-8bbd-338fdb95ddb7 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:00.558515 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-41ab367c-1bf6-4282-8bbd-338fdb95ddb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:00.586639 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ab367c-1bf6-4282-8bbd-338fdb95ddb7 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:00.671582 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47814a4a-0475-4030-80dc-fa5c26b2e324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:00.672543 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47814a4a-0475-4030-80dc-fa5c26b2e324 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:00.672825 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47814a4a-0475-4030-80dc-fa5c26b2e324 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:00.673372 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47814a4a-0475-4030-80dc-fa5c26b2e324 admin admin] PATH_INFO: `/v3/projects/8eb04aedc9b84408ad7405342b0a92ad` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:00.674615 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47814a4a-0475-4030-80dc-fa5c26b2e324 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8eb04aedc9b84408ad7405342b0a92ad)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:00.676850 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47814a4a-0475-4030-80dc-fa5c26b2e324 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:00.804706 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-47814a4a-0475-4030-80dc-fa5c26b2e324 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8eb04aedc9b84408ad7405342b0a92ad', 'request_id': 'req-47814a4a-0475-4030-80dc-fa5c26b2e324'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:00.816002 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-47814a4a-0475-4030-80dc-fa5c26b2e324 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8eb04aedc9b84408ad7405342b0a92ad', 'request_id': 'req-47814a4a-0475-4030-80dc-fa5c26b2e324'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:00.823375 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-47814a4a-0475-4030-80dc-fa5c26b2e324 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8eb04aedc9b84408ad7405342b0a92ad', 'request_id': 'req-47814a4a-0475-4030-80dc-fa5c26b2e324'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:00.923246 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-47814a4a-0475-4030-80dc-fa5c26b2e324 admin admin] The token cache is being invalidate because project 8eb04aedc9b84408ad7405342b0a92ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:04:00.923661 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-47814a4a-0475-4030-80dc-fa5c26b2e324 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:00.924383 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-47814a4a-0475-4030-80dc-fa5c26b2e324 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:00.926083 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 897/1775] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:04:00 2022] DELETE /identity/v3/projects/8eb04aedc9b84408ad7405342b0a92ad => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:04:00.931144 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a94d390-add4-4875-ae0d-2cd8579889af None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:00.932224 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a94d390-add4-4875-ae0d-2cd8579889af None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:00.932325 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a94d390-add4-4875-ae0d-2cd8579889af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:00.953173 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-1a94d390-add4-4875-ae0d-2cd8579889af None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:04:00.954523 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-1a94d390-add4-4875-ae0d-2cd8579889af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:01.030892 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 879/1776] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:04:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:04:01.046335 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a62e10-a018-461a-9ce9-3d1441a6cdc3 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:01.046550 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a62e10-a018-461a-9ce9-3d1441a6cdc3 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:01.046837 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a62e10-a018-461a-9ce9-3d1441a6cdc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:01.067723 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-a1a62e10-a018-461a-9ce9-3d1441a6cdc3 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:04:01.068986 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a1a62e10-a018-461a-9ce9-3d1441a6cdc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:01.080281 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 898/1777] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:04:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:04:01.085160 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a94d390-add4-4875-ae0d-2cd8579889af None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:01.099164 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a94d390-add4-4875-ae0d-2cd8579889af None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:01.105592 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff74255f-6df2-444e-818d-b9de4aaebb5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:01.106480 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff74255f-6df2-444e-818d-b9de4aaebb5f admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:01.108224 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff74255f-6df2-444e-818d-b9de4aaebb5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:01.108519 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff74255f-6df2-444e-818d-b9de4aaebb5f admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:01.109354 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff74255f-6df2-444e-818d-b9de4aaebb5f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 13:04:01.109675 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff74255f-6df2-444e-818d-b9de4aaebb5f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:01.111312 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff74255f-6df2-444e-818d-b9de4aaebb5f admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:01.126288 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 880/1778] 10.222.0.27 () {58 vars in 1152 bytes} [Fri Jun 10 13:04:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 13:04:01.144109 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a62e10-a018-461a-9ce9-3d1441a6cdc3 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:01.158764 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a62e10-a018-461a-9ce9-3d1441a6cdc3 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:01.166809 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b731dd54-e1e9-4092-8bce-1a00c42ec7f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:01.167898 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b731dd54-e1e9-4092-8bce-1a00c42ec7f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:01.168355 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b731dd54-e1e9-4092-8bce-1a00c42ec7f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:01.168976 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b731dd54-e1e9-4092-8bce-1a00c42ec7f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:01.170190 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b731dd54-e1e9-4092-8bce-1a00c42ec7f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:01.174240 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b731dd54-e1e9-4092-8bce-1a00c42ec7f8 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:01.320361 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 899/1779] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:04:01 2022] POST /identity/v3/projects => generated 367 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:04:01.321967 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff74255f-6df2-444e-818d-b9de4aaebb5f admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:01.333813 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff74255f-6df2-444e-818d-b9de4aaebb5f admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:01.420461 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01df2e40-072c-4519-92ef-4da84947d82d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:01.424224 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01df2e40-072c-4519-92ef-4da84947d82d admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:01.424224 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01df2e40-072c-4519-92ef-4da84947d82d admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:01.424224 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01df2e40-072c-4519-92ef-4da84947d82d admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:01.424224 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01df2e40-072c-4519-92ef-4da84947d82d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:01.424876 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01df2e40-072c-4519-92ef-4da84947d82d admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:01.527831 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 881/1780] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:04:01 2022] POST /identity/v3/users => generated 284 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:04:01.532835 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b731dd54-e1e9-4092-8bce-1a00c42ec7f8 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:01.544223 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b731dd54-e1e9-4092-8bce-1a00c42ec7f8 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:01.549095 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce951674-17f7-452a-be8f-42f3bff058fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:01.549785 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce951674-17f7-452a-be8f-42f3bff058fc admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:01.550081 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce951674-17f7-452a-be8f-42f3bff058fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:01.550456 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce951674-17f7-452a-be8f-42f3bff058fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:01.551102 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce951674-17f7-452a-be8f-42f3bff058fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:01.552968 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce951674-17f7-452a-be8f-42f3bff058fc admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:01.571898 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 900/1781] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:04:01 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:04:01.575318 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01df2e40-072c-4519-92ef-4da84947d82d admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:01.587743 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01df2e40-072c-4519-92ef-4da84947d82d admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:01.594572 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a87346-85c1-4d8f-9811-d5cfe87b53a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:01.595057 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a87346-85c1-4d8f-9811-d5cfe87b53a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:01.595342 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a87346-85c1-4d8f-9811-d5cfe87b53a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:01.595618 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a87346-85c1-4d8f-9811-d5cfe87b53a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:01.597304 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a87346-85c1-4d8f-9811-d5cfe87b53a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:01.599016 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a87346-85c1-4d8f-9811-d5cfe87b53a9 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:01.616736 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 882/1782] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:04:01 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:04:01.617452 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce951674-17f7-452a-be8f-42f3bff058fc admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:01.632177 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce951674-17f7-452a-be8f-42f3bff058fc admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:01.662655 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c6c70a-d244-4580-8c62-01d74754cbee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:01.664611 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c6c70a-d244-4580-8c62-01d74754cbee admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:01.664925 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c6c70a-d244-4580-8c62-01d74754cbee admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:01.665280 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c6c70a-d244-4580-8c62-01d74754cbee admin admin] PATH_INFO: `/v3/projects/47f6827158254b258c5199b10cdb3656/users/03072bb3844d410db4d20a8e29f62963/roles/43106c664e1b40de984c0c62701149c8` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:01.690731 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c6c70a-d244-4580-8c62-01d74754cbee admin admin] RBAC: Authorizing `identity:create_grant(project_id=47f6827158254b258c5199b10cdb3656, user_id=03072bb3844d410db4d20a8e29f62963, role_id=43106c664e1b40de984c0c62701149c8)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:01.694430 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c6c70a-d244-4580-8c62-01d74754cbee admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:01.810370 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 901/1783] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:04:01 2022] PUT /identity/v3/projects/47f6827158254b258c5199b10cdb3656/users/03072bb3844d410db4d20a8e29f62963/roles/43106c664e1b40de984c0c62701149c8 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:04:01.820701 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d74a7f-0056-4621-9cba-73912c8d84cb None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:01.821046 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d74a7f-0056-4621-9cba-73912c8d84cb None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:01.821327 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d74a7f-0056-4621-9cba-73912c8d84cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:01.849051 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-64d74a7f-0056-4621-9cba-73912c8d84cb None None] MFA Rules not processed for user `03072bb3844d410db4d20a8e29f62963`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:04:01.850528 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-64d74a7f-0056-4621-9cba-73912c8d84cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:01.877570 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:04:01.918506 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 883/1784] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:04:01 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:04:01.923375 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c6c70a-d244-4580-8c62-01d74754cbee admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:01.946136 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:04:01.995159 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c6c70a-d244-4580-8c62-01d74754cbee admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:02.000209 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973f06ec-5b6b-4765-803b-3ccd09984354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:02.001102 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973f06ec-5b6b-4765-803b-3ccd09984354 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:02.001570 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973f06ec-5b6b-4765-803b-3ccd09984354 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:02.001848 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973f06ec-5b6b-4765-803b-3ccd09984354 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:02.002670 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973f06ec-5b6b-4765-803b-3ccd09984354 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:02.004616 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973f06ec-5b6b-4765-803b-3ccd09984354 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:02.130669 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 902/1785] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:04:01 2022] POST /identity/v3/projects => generated 367 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 10 13:04:02.130933 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d74a7f-0056-4621-9cba-73912c8d84cb None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:02.142859 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d74a7f-0056-4621-9cba-73912c8d84cb None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:02.202566 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca89af35-7e68-424b-8cac-544aa71a374d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:02.203242 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca89af35-7e68-424b-8cac-544aa71a374d admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:02.203530 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca89af35-7e68-424b-8cac-544aa71a374d admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:02.203809 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca89af35-7e68-424b-8cac-544aa71a374d admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:02.204548 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca89af35-7e68-424b-8cac-544aa71a374d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:02.206466 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca89af35-7e68-424b-8cac-544aa71a374d admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:02.351815 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 884/1786] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:04:02 2022] POST /identity/v3/users => generated 290 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:04:02.356438 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973f06ec-5b6b-4765-803b-3ccd09984354 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:02.366608 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973f06ec-5b6b-4765-803b-3ccd09984354 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:02.371696 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ebbd0d-f3be-4472-ab39-9adfd5bd58ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:02.372380 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ebbd0d-f3be-4472-ab39-9adfd5bd58ce admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:02.372670 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ebbd0d-f3be-4472-ab39-9adfd5bd58ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:02.372999 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ebbd0d-f3be-4472-ab39-9adfd5bd58ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:02.373743 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ebbd0d-f3be-4472-ab39-9adfd5bd58ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:02.375317 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ebbd0d-f3be-4472-ab39-9adfd5bd58ce admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:02.387158 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 903/1787] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:04:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:04:02.397746 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca89af35-7e68-424b-8cac-544aa71a374d admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:02.402289 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca89af35-7e68-424b-8cac-544aa71a374d admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:02.411719 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b676c0-3d0f-4ed4-8b84-9b76a15828f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:02.411719 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b676c0-3d0f-4ed4-8b84-9b76a15828f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:02.411719 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b676c0-3d0f-4ed4-8b84-9b76a15828f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:02.411719 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b676c0-3d0f-4ed4-8b84-9b76a15828f3 admin admin] PATH_INFO: `/v3/projects/bf6bfcb05bed499eabcb77cb8b275ffe/users/3b1e85c0c4994e8e9c7d81c512668996/roles/b43d5a8247e244869c2c70f986f42e8a` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:02.436514 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b676c0-3d0f-4ed4-8b84-9b76a15828f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf6bfcb05bed499eabcb77cb8b275ffe, user_id=3b1e85c0c4994e8e9c7d81c512668996, role_id=b43d5a8247e244869c2c70f986f42e8a)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:02.441359 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b676c0-3d0f-4ed4-8b84-9b76a15828f3 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:02.525916 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 885/1788] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:04:02 2022] PUT /identity/v3/projects/bf6bfcb05bed499eabcb77cb8b275ffe/users/3b1e85c0c4994e8e9c7d81c512668996/roles/b43d5a8247e244869c2c70f986f42e8a => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:04:02.530452 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ebbd0d-f3be-4472-ab39-9adfd5bd58ce admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:02.541950 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ebbd0d-f3be-4472-ab39-9adfd5bd58ce admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:02.599675 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74b2c63-d82a-4c05-b94f-1e50f13412a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:02.600406 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74b2c63-d82a-4c05-b94f-1e50f13412a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:02.600692 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74b2c63-d82a-4c05-b94f-1e50f13412a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:02.600985 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74b2c63-d82a-4c05-b94f-1e50f13412a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:02.601746 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74b2c63-d82a-4c05-b94f-1e50f13412a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:02.604649 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74b2c63-d82a-4c05-b94f-1e50f13412a6 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:02.613901 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 904/1789] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:04:02 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 10 13:04:02.618023 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b676c0-3d0f-4ed4-8b84-9b76a15828f3 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:02.628231 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b676c0-3d0f-4ed4-8b84-9b76a15828f3 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:02.632808 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce74111-7ffd-444d-b691-ca4ddc9d8b9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:02.633681 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce74111-7ffd-444d-b691-ca4ddc9d8b9c admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:02.633964 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce74111-7ffd-444d-b691-ca4ddc9d8b9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:02.634243 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce74111-7ffd-444d-b691-ca4ddc9d8b9c admin admin] PATH_INFO: `/v3/projects/bf6bfcb05bed499eabcb77cb8b275ffe/users/3b1e85c0c4994e8e9c7d81c512668996/roles/b43d5a8247e244869c2c70f986f42e8a` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:02.636263 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce74111-7ffd-444d-b691-ca4ddc9d8b9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf6bfcb05bed499eabcb77cb8b275ffe, user_id=3b1e85c0c4994e8e9c7d81c512668996, role_id=b43d5a8247e244869c2c70f986f42e8a)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:02.638847 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce74111-7ffd-444d-b691-ca4ddc9d8b9c admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:02.649354 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 886/1790] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:04:02 2022] PUT /identity/v3/projects/bf6bfcb05bed499eabcb77cb8b275ffe/users/3b1e85c0c4994e8e9c7d81c512668996/roles/b43d5a8247e244869c2c70f986f42e8a => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:04:02.662104 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c84cb3-bf1a-4772-97a3-e213cc86d873 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:02.662400 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c84cb3-bf1a-4772-97a3-e213cc86d873 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:02.662681 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c84cb3-bf1a-4772-97a3-e213cc86d873 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:02.689832 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-11c84cb3-bf1a-4772-97a3-e213cc86d873 None None] MFA Rules not processed for user `3b1e85c0c4994e8e9c7d81c512668996`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:04:02.691028 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-11c84cb3-bf1a-4772-97a3-e213cc86d873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:02.774539 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 905/1791] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:04:02 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 10 13:04:02.801311 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce74111-7ffd-444d-b691-ca4ddc9d8b9c admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:02.803673 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-bce74111-7ffd-444d-b691-ca4ddc9d8b9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:02.862105 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce74111-7ffd-444d-b691-ca4ddc9d8b9c admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:02.866085 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eced567f-e857-4912-bd42-68902985481d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6cc95a563b941ac92ab283aa0d5a5d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:02.866732 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eced567f-e857-4912-bd42-68902985481d service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:02.867010 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eced567f-e857-4912-bd42-68902985481d service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:02.867281 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eced567f-e857-4912-bd42-68902985481d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:02.876544 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eced567f-e857-4912-bd42-68902985481d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:02.877957 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eced567f-e857-4912-bd42-68902985481d service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:02.888254 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 887/1792] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:04:02 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:04:06.747206 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c84cb3-bf1a-4772-97a3-e213cc86d873 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:06.780512 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:04:06.851267 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c84cb3-bf1a-4772-97a3-e213cc86d873 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:06.856645 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9251b5f5-1dc6-4822-8a6c-8c3efdf82f12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:06.857789 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9251b5f5-1dc6-4822-8a6c-8c3efdf82f12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:06.858077 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9251b5f5-1dc6-4822-8a6c-8c3efdf82f12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:06.858613 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9251b5f5-1dc6-4822-8a6c-8c3efdf82f12 admin admin] PATH_INFO: `/v3/users/03072bb3844d410db4d20a8e29f62963` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:06.859917 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9251b5f5-1dc6-4822-8a6c-8c3efdf82f12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=03072bb3844d410db4d20a8e29f62963)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:06.862484 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9251b5f5-1dc6-4822-8a6c-8c3efdf82f12 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:07.083695 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-9251b5f5-1dc6-4822-8a6c-8c3efdf82f12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03072bb3844d410db4d20a8e29f62963'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:07.092260 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-9251b5f5-1dc6-4822-8a6c-8c3efdf82f12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03072bb3844d410db4d20a8e29f62963'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:07.211177 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-9251b5f5-1dc6-4822-8a6c-8c3efdf82f12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03072bb3844d410db4d20a8e29f62963'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:07.212026 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-9251b5f5-1dc6-4822-8a6c-8c3efdf82f12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03072bb3844d410db4d20a8e29f62963'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:07.226928 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-9251b5f5-1dc6-4822-8a6c-8c3efdf82f12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03072bb3844d410db4d20a8e29f62963'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:07.233757 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 906/1793] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:04:06 2022] DELETE /identity/v3/users/03072bb3844d410db4d20a8e29f62963 => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:04:07.238438 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9888a0b2-1eac-43e6-9ad7-897daa5a0b7a None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:07.238914 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9888a0b2-1eac-43e6-9ad7-897daa5a0b7a None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:07.238992 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9888a0b2-1eac-43e6-9ad7-897daa5a0b7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:07.260514 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-9888a0b2-1eac-43e6-9ad7-897daa5a0b7a None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:04:07.262010 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-9888a0b2-1eac-43e6-9ad7-897daa5a0b7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:07.282572 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:04:07.325914 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 888/1794] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:04:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:04:07.336226 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9251b5f5-1dc6-4822-8a6c-8c3efdf82f12 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:07.338209 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9251b5f5-1dc6-4822-8a6c-8c3efdf82f12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:07.395167 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9251b5f5-1dc6-4822-8a6c-8c3efdf82f12 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:07.398779 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2190822c-8f3e-4322-ba3d-5488e7b2c3bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:07.399691 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2190822c-8f3e-4322-ba3d-5488e7b2c3bb service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:07.400658 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2190822c-8f3e-4322-ba3d-5488e7b2c3bb service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:07.400981 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2190822c-8f3e-4322-ba3d-5488e7b2c3bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:07.410340 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2190822c-8f3e-4322-ba3d-5488e7b2c3bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:07.411758 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2190822c-8f3e-4322-ba3d-5488e7b2c3bb service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:07.421557 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 907/1795] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:04:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:04:07.854217 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9888a0b2-1eac-43e6-9ad7-897daa5a0b7a None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:07.859253 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-9888a0b2-1eac-43e6-9ad7-897daa5a0b7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:07.888577 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9888a0b2-1eac-43e6-9ad7-897daa5a0b7a None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:07.894443 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b05b49-a2ae-4eb2-a869-a899c25552b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:07.895325 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b05b49-a2ae-4eb2-a869-a899c25552b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:07.895622 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b05b49-a2ae-4eb2-a869-a899c25552b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:07.895956 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b05b49-a2ae-4eb2-a869-a899c25552b9 admin admin] PATH_INFO: `/v3/projects/47f6827158254b258c5199b10cdb3656` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:07.897240 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b05b49-a2ae-4eb2-a869-a899c25552b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=47f6827158254b258c5199b10cdb3656)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:07.902350 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b05b49-a2ae-4eb2-a869-a899c25552b9 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:08.013316 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-35b05b49-a2ae-4eb2-a869-a899c25552b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47f6827158254b258c5199b10cdb3656', 'request_id': 'req-35b05b49-a2ae-4eb2-a869-a899c25552b9'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:08.024702 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-35b05b49-a2ae-4eb2-a869-a899c25552b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47f6827158254b258c5199b10cdb3656', 'request_id': 'req-35b05b49-a2ae-4eb2-a869-a899c25552b9'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:08.032066 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-35b05b49-a2ae-4eb2-a869-a899c25552b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47f6827158254b258c5199b10cdb3656', 'request_id': 'req-35b05b49-a2ae-4eb2-a869-a899c25552b9'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:08.135977 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-35b05b49-a2ae-4eb2-a869-a899c25552b9 admin admin] The token cache is being invalidate because project 47f6827158254b258c5199b10cdb3656 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:04:08.136475 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-35b05b49-a2ae-4eb2-a869-a899c25552b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:08.137330 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-35b05b49-a2ae-4eb2-a869-a899c25552b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:08.139091 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 889/1796] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:04:07 2022] DELETE /identity/v3/projects/47f6827158254b258c5199b10cdb3656 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:04:08.143103 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2190822c-8f3e-4322-ba3d-5488e7b2c3bb service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:08.155698 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2190822c-8f3e-4322-ba3d-5488e7b2c3bb service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:08.202359 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:04:08.239503 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f7be2f-69be-452e-8209-48aec2f506dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:08.241272 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f7be2f-69be-452e-8209-48aec2f506dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:08.241888 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f7be2f-69be-452e-8209-48aec2f506dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:08.242647 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f7be2f-69be-452e-8209-48aec2f506dd admin admin] PATH_INFO: `/v3/users/3b1e85c0c4994e8e9c7d81c512668996` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:08.246567 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f7be2f-69be-452e-8209-48aec2f506dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b1e85c0c4994e8e9c7d81c512668996)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:08.250919 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f7be2f-69be-452e-8209-48aec2f506dd admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:08.435263 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b4f7be2f-69be-452e-8209-48aec2f506dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b1e85c0c4994e8e9c7d81c512668996'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:08.443038 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b4f7be2f-69be-452e-8209-48aec2f506dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b1e85c0c4994e8e9c7d81c512668996'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:08.547095 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b4f7be2f-69be-452e-8209-48aec2f506dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b1e85c0c4994e8e9c7d81c512668996'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:08.547748 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b4f7be2f-69be-452e-8209-48aec2f506dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b1e85c0c4994e8e9c7d81c512668996'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:08.560071 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b4f7be2f-69be-452e-8209-48aec2f506dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b1e85c0c4994e8e9c7d81c512668996'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:08.565918 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 908/1797] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:04:08 2022] DELETE /identity/v3/users/3b1e85c0c4994e8e9c7d81c512668996 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:04:08.977850 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b05b49-a2ae-4eb2-a869-a899c25552b9 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:08.979421 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-35b05b49-a2ae-4eb2-a869-a899c25552b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:08.997299 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b05b49-a2ae-4eb2-a869-a899c25552b9 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:09.058080 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ce1332-558e-4a43-bfd0-c1585faf6be3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:09.058949 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ce1332-558e-4a43-bfd0-c1585faf6be3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:09.059230 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ce1332-558e-4a43-bfd0-c1585faf6be3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:09.059500 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ce1332-558e-4a43-bfd0-c1585faf6be3 admin admin] PATH_INFO: `/v3/projects/bf6bfcb05bed499eabcb77cb8b275ffe` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:09.060851 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ce1332-558e-4a43-bfd0-c1585faf6be3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf6bfcb05bed499eabcb77cb8b275ffe)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:09.062853 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ce1332-558e-4a43-bfd0-c1585faf6be3 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:09.196308 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:04:09.198624 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-07ce1332-558e-4a43-bfd0-c1585faf6be3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf6bfcb05bed499eabcb77cb8b275ffe', 'request_id': 'req-07ce1332-558e-4a43-bfd0-c1585faf6be3'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:09.204143 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-07ce1332-558e-4a43-bfd0-c1585faf6be3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf6bfcb05bed499eabcb77cb8b275ffe', 'request_id': 'req-07ce1332-558e-4a43-bfd0-c1585faf6be3'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:09.212964 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-07ce1332-558e-4a43-bfd0-c1585faf6be3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf6bfcb05bed499eabcb77cb8b275ffe', 'request_id': 'req-07ce1332-558e-4a43-bfd0-c1585faf6be3'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:09.320056 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-07ce1332-558e-4a43-bfd0-c1585faf6be3 admin admin] The token cache is being invalidate because project bf6bfcb05bed499eabcb77cb8b275ffe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:04:09.320352 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-07ce1332-558e-4a43-bfd0-c1585faf6be3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:09.322873 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-07ce1332-558e-4a43-bfd0-c1585faf6be3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:09.325295 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 890/1798] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:04:08 2022] DELETE /identity/v3/projects/bf6bfcb05bed499eabcb77cb8b275ffe => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:04:09.330185 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21075f0-3640-4242-8942-06c0737161f9 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:09.339992 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21075f0-3640-4242-8942-06c0737161f9 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:09.339992 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21075f0-3640-4242-8942-06c0737161f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:09.348852 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-b21075f0-3640-4242-8942-06c0737161f9 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:04:09.350048 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b21075f0-3640-4242-8942-06c0737161f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:09.413393 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 909/1799] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:04:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:04:09.421156 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebdcba5-db85-47ed-b612-539030ba8c82 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:09.421457 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebdcba5-db85-47ed-b612-539030ba8c82 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:09.421742 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebdcba5-db85-47ed-b612-539030ba8c82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:09.440397 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-3ebdcba5-db85-47ed-b612-539030ba8c82 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:04:09.441596 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-3ebdcba5-db85-47ed-b612-539030ba8c82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:09.450074 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 891/1800] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:04:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:04:09.454414 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21075f0-3640-4242-8942-06c0737161f9 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:09.464077 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21075f0-3640-4242-8942-06c0737161f9 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:09.468909 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa36ee7a-9109-4d61-9f88-8effc3788b93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:09.469652 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa36ee7a-9109-4d61-9f88-8effc3788b93 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:09.469934 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa36ee7a-9109-4d61-9f88-8effc3788b93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:09.470231 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa36ee7a-9109-4d61-9f88-8effc3788b93 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:09.470958 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa36ee7a-9109-4d61-9f88-8effc3788b93 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 13:04:09.471310 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa36ee7a-9109-4d61-9f88-8effc3788b93 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:09.473078 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa36ee7a-9109-4d61-9f88-8effc3788b93 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:09.490932 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 910/1801] 10.222.0.27 () {58 vars in 1152 bytes} [Fri Jun 10 13:04:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 13:04:09.491300 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebdcba5-db85-47ed-b612-539030ba8c82 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:09.501146 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebdcba5-db85-47ed-b612-539030ba8c82 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:09.506301 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb77e20b-0fc0-449c-879f-335438c90bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:09.506957 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb77e20b-0fc0-449c-879f-335438c90bc6 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:09.507239 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb77e20b-0fc0-449c-879f-335438c90bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:09.507898 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb77e20b-0fc0-449c-879f-335438c90bc6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:09.508264 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb77e20b-0fc0-449c-879f-335438c90bc6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:09.515967 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb77e20b-0fc0-449c-879f-335438c90bc6 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:09.589271 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 892/1802] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:04:09 2022] POST /identity/v3/projects => generated 381 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:04:09.593322 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa36ee7a-9109-4d61-9f88-8effc3788b93 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:09.606467 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa36ee7a-9109-4d61-9f88-8effc3788b93 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:09.662376 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d29a4c8-2417-4234-85eb-fcffc02de877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:09.663044 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d29a4c8-2417-4234-85eb-fcffc02de877 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:09.663329 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d29a4c8-2417-4234-85eb-fcffc02de877 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:09.663608 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d29a4c8-2417-4234-85eb-fcffc02de877 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:09.664331 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d29a4c8-2417-4234-85eb-fcffc02de877 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:09.666780 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d29a4c8-2417-4234-85eb-fcffc02de877 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:09.766978 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 911/1803] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:04:09 2022] POST /identity/v3/users => generated 291 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:04:09.770812 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb77e20b-0fc0-449c-879f-335438c90bc6 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:09.782480 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb77e20b-0fc0-449c-879f-335438c90bc6 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:09.789381 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d4caaf-d2cd-45c1-b586-1c7e02d0373a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:09.790057 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d4caaf-d2cd-45c1-b586-1c7e02d0373a admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:09.790915 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d4caaf-d2cd-45c1-b586-1c7e02d0373a admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:09.791259 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d4caaf-d2cd-45c1-b586-1c7e02d0373a admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:09.793514 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d4caaf-d2cd-45c1-b586-1c7e02d0373a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:09.793800 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d4caaf-d2cd-45c1-b586-1c7e02d0373a admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:09.806892 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 893/1804] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:04:09 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:04:09.810853 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d29a4c8-2417-4234-85eb-fcffc02de877 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:09.822183 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d29a4c8-2417-4234-85eb-fcffc02de877 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:09.828668 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab82d699-f53c-4a6a-88c0-0b9dd458579d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:09.829584 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab82d699-f53c-4a6a-88c0-0b9dd458579d admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:09.831499 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab82d699-f53c-4a6a-88c0-0b9dd458579d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:09.831499 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab82d699-f53c-4a6a-88c0-0b9dd458579d admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:09.831499 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab82d699-f53c-4a6a-88c0-0b9dd458579d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:09.833357 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab82d699-f53c-4a6a-88c0-0b9dd458579d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:09.854987 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 912/1805] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:04:09 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 10 13:04:09.859228 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d4caaf-d2cd-45c1-b586-1c7e02d0373a admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:09.871601 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d4caaf-d2cd-45c1-b586-1c7e02d0373a admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:09.876613 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f94b110-99ac-4a56-ba46-a0d332bbbb7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:09.877511 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f94b110-99ac-4a56-ba46-a0d332bbbb7f admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:09.877795 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f94b110-99ac-4a56-ba46-a0d332bbbb7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:09.878071 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f94b110-99ac-4a56-ba46-a0d332bbbb7f admin admin] PATH_INFO: `/v3/projects/3c027d3ab445467aa62d8083c7b3bbec/users/a0ae8699d725402ba626a58f22f87f6b/roles/43106c664e1b40de984c0c62701149c8` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:09.893075 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f94b110-99ac-4a56-ba46-a0d332bbbb7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c027d3ab445467aa62d8083c7b3bbec, user_id=a0ae8699d725402ba626a58f22f87f6b, role_id=43106c664e1b40de984c0c62701149c8)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:09.895899 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f94b110-99ac-4a56-ba46-a0d332bbbb7f admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:09.964596 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 894/1806] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:04:09 2022] PUT /identity/v3/projects/3c027d3ab445467aa62d8083c7b3bbec/users/a0ae8699d725402ba626a58f22f87f6b/roles/43106c664e1b40de984c0c62701149c8 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:04:09.974842 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77884df-74f8-4b9b-953a-790d483b2d22 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:09.975140 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77884df-74f8-4b9b-953a-790d483b2d22 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:09.975480 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77884df-74f8-4b9b-953a-790d483b2d22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:10.003272 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-c77884df-74f8-4b9b-953a-790d483b2d22 None None] MFA Rules not processed for user `a0ae8699d725402ba626a58f22f87f6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:04:10.004634 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c77884df-74f8-4b9b-953a-790d483b2d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:10.088581 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 913/1807] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:04:09 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:04:10.093504 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f94b110-99ac-4a56-ba46-a0d332bbbb7f admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:10.153844 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f94b110-99ac-4a56-ba46-a0d332bbbb7f admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:10.158510 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67629907-4be1-4c1b-9d01-e2faa230c5a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:10.159161 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67629907-4be1-4c1b-9d01-e2faa230c5a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:10.159449 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67629907-4be1-4c1b-9d01-e2faa230c5a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:10.159730 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67629907-4be1-4c1b-9d01-e2faa230c5a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:10.160492 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67629907-4be1-4c1b-9d01-e2faa230c5a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:10.168565 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67629907-4be1-4c1b-9d01-e2faa230c5a6 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:10.296129 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 895/1808] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:04:10 2022] POST /identity/v3/projects => generated 379 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 10 13:04:10.297681 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77884df-74f8-4b9b-953a-790d483b2d22 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:10.308771 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77884df-74f8-4b9b-953a-790d483b2d22 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:10.385640 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e170cb06-d843-4602-8c43-710d791e3d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:10.388462 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e170cb06-d843-4602-8c43-710d791e3d48 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:10.388462 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e170cb06-d843-4602-8c43-710d791e3d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:10.388462 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e170cb06-d843-4602-8c43-710d791e3d48 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:10.388462 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e170cb06-d843-4602-8c43-710d791e3d48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:10.390641 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e170cb06-d843-4602-8c43-710d791e3d48 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:10.515706 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 914/1809] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:04:10 2022] POST /identity/v3/users => generated 296 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:04:10.529195 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67629907-4be1-4c1b-9d01-e2faa230c5a6 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:10.566296 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67629907-4be1-4c1b-9d01-e2faa230c5a6 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:10.572281 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ada8596-0911-43c0-9e5e-907d17cccba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:10.572961 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ada8596-0911-43c0-9e5e-907d17cccba6 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:10.573238 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ada8596-0911-43c0-9e5e-907d17cccba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:10.573512 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ada8596-0911-43c0-9e5e-907d17cccba6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:10.574509 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ada8596-0911-43c0-9e5e-907d17cccba6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:10.576560 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ada8596-0911-43c0-9e5e-907d17cccba6 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:10.590381 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 896/1810] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:04:10 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:04:10.594394 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e170cb06-d843-4602-8c43-710d791e3d48 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:10.606121 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e170cb06-d843-4602-8c43-710d791e3d48 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:10.612286 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c432e0d-1729-4da5-a7da-37bca4b962a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:10.613223 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c432e0d-1729-4da5-a7da-37bca4b962a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:10.613991 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c432e0d-1729-4da5-a7da-37bca4b962a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:10.614302 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c432e0d-1729-4da5-a7da-37bca4b962a5 admin admin] PATH_INFO: `/v3/projects/9bf0a077d9ba436f9a74a68b57a88f61/users/f079cda0e8db4d619a56b0a208f56bbb/roles/b43d5a8247e244869c2c70f986f42e8a` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:10.618263 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:04:10.631747 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c432e0d-1729-4da5-a7da-37bca4b962a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bf0a077d9ba436f9a74a68b57a88f61, user_id=f079cda0e8db4d619a56b0a208f56bbb, role_id=b43d5a8247e244869c2c70f986f42e8a)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:10.634990 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c432e0d-1729-4da5-a7da-37bca4b962a5 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:10.707402 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 915/1811] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:04:10 2022] PUT /identity/v3/projects/9bf0a077d9ba436f9a74a68b57a88f61/users/f079cda0e8db4d619a56b0a208f56bbb/roles/b43d5a8247e244869c2c70f986f42e8a => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:04:10.714080 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ada8596-0911-43c0-9e5e-907d17cccba6 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:10.724450 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ada8596-0911-43c0-9e5e-907d17cccba6 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:10.784088 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c064e0b-b11d-482a-9ad0-60786b1ec0fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:10.784770 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c064e0b-b11d-482a-9ad0-60786b1ec0fc admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:10.785056 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c064e0b-b11d-482a-9ad0-60786b1ec0fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:10.785338 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c064e0b-b11d-482a-9ad0-60786b1ec0fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:10.786071 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c064e0b-b11d-482a-9ad0-60786b1ec0fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:10.787671 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c064e0b-b11d-482a-9ad0-60786b1ec0fc admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:10.797849 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 897/1812] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:04:10 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:04:10.802081 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c432e0d-1729-4da5-a7da-37bca4b962a5 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:10.813011 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c432e0d-1729-4da5-a7da-37bca4b962a5 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:10.817726 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4552bfef-a81f-414a-8cd5-ed67d645d9e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:10.818622 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4552bfef-a81f-414a-8cd5-ed67d645d9e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:10.818894 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4552bfef-a81f-414a-8cd5-ed67d645d9e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:10.819394 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4552bfef-a81f-414a-8cd5-ed67d645d9e3 admin admin] PATH_INFO: `/v3/projects/9bf0a077d9ba436f9a74a68b57a88f61/users/f079cda0e8db4d619a56b0a208f56bbb/roles/b43d5a8247e244869c2c70f986f42e8a` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:10.821276 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4552bfef-a81f-414a-8cd5-ed67d645d9e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bf0a077d9ba436f9a74a68b57a88f61, user_id=f079cda0e8db4d619a56b0a208f56bbb, role_id=b43d5a8247e244869c2c70f986f42e8a)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:10.824499 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4552bfef-a81f-414a-8cd5-ed67d645d9e3 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:10.835464 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 916/1813] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:04:10 2022] PUT /identity/v3/projects/9bf0a077d9ba436f9a74a68b57a88f61/users/f079cda0e8db4d619a56b0a208f56bbb/roles/b43d5a8247e244869c2c70f986f42e8a => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:04:10.848074 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234b50a5-a809-4f83-95d3-703c8df8be13 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:10.848074 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234b50a5-a809-4f83-95d3-703c8df8be13 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:10.848074 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234b50a5-a809-4f83-95d3-703c8df8be13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:10.880427 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-234b50a5-a809-4f83-95d3-703c8df8be13 None None] MFA Rules not processed for user `f079cda0e8db4d619a56b0a208f56bbb`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:04:10.882893 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-234b50a5-a809-4f83-95d3-703c8df8be13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:10.888548 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4552bfef-a81f-414a-8cd5-ed67d645d9e3 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:10.890222 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-4552bfef-a81f-414a-8cd5-ed67d645d9e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:10.948856 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 898/1814] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:04:10 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:04:10.955925 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4552bfef-a81f-414a-8cd5-ed67d645d9e3 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:10.960956 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234b50a5-a809-4f83-95d3-703c8df8be13 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:10.961084 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec86103-bd6e-4169-873a-3f6c2ede4e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:10.961859 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec86103-bd6e-4169-873a-3f6c2ede4e5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:10.962172 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec86103-bd6e-4169-873a-3f6c2ede4e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:10.962456 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec86103-bd6e-4169-873a-3f6c2ede4e5d admin admin] PATH_INFO: `/v3/users/05108a8d34ca4060b5a4c18b24556fba` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:10.962789 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-234b50a5-a809-4f83-95d3-703c8df8be13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:10.963664 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec86103-bd6e-4169-873a-3f6c2ede4e5d admin admin] RBAC: Authorizing `identity:delete_user(user_id=05108a8d34ca4060b5a4c18b24556fba)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:10.965643 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec86103-bd6e-4169-873a-3f6c2ede4e5d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:11.019027 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234b50a5-a809-4f83-95d3-703c8df8be13 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:11.022716 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94118fb2-2f27-4602-a0b5-4b1e4a40551b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6cc95a563b941ac92ab283aa0d5a5d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:11.023361 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94118fb2-2f27-4602-a0b5-4b1e4a40551b service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:11.023643 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94118fb2-2f27-4602-a0b5-4b1e4a40551b service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:11.024239 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94118fb2-2f27-4602-a0b5-4b1e4a40551b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:11.033188 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94118fb2-2f27-4602-a0b5-4b1e4a40551b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:11.034613 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94118fb2-2f27-4602-a0b5-4b1e4a40551b service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:11.044067 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 899/1815] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:04:10 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 10 13:04:11.175990 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-9ec86103-bd6e-4169-873a-3f6c2ede4e5d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05108a8d34ca4060b5a4c18b24556fba'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:11.183137 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-9ec86103-bd6e-4169-873a-3f6c2ede4e5d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05108a8d34ca4060b5a4c18b24556fba'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:11.322443 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-9ec86103-bd6e-4169-873a-3f6c2ede4e5d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05108a8d34ca4060b5a4c18b24556fba'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:11.323067 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-9ec86103-bd6e-4169-873a-3f6c2ede4e5d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05108a8d34ca4060b5a4c18b24556fba'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:11.335709 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-9ec86103-bd6e-4169-873a-3f6c2ede4e5d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05108a8d34ca4060b5a4c18b24556fba'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:11.341669 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 917/1816] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:04:10 2022] DELETE /identity/v3/users/05108a8d34ca4060b5a4c18b24556fba => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 10 13:04:11.346296 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07561ece-d45d-4906-9f76-f054d2b705c3 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:11.346772 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07561ece-d45d-4906-9f76-f054d2b705c3 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:11.346880 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07561ece-d45d-4906-9f76-f054d2b705c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:11.364246 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-07561ece-d45d-4906-9f76-f054d2b705c3 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:04:11.365367 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-07561ece-d45d-4906-9f76-f054d2b705c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:11.432325 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 900/1817] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:04:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:04:11.441976 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec86103-bd6e-4169-873a-3f6c2ede4e5d admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:11.443494 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9ec86103-bd6e-4169-873a-3f6c2ede4e5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:11.506770 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec86103-bd6e-4169-873a-3f6c2ede4e5d admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:11.510463 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf4b447-a823-4a1d-b9c6-6cbf90cbad73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:11.511116 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf4b447-a823-4a1d-b9c6-6cbf90cbad73 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:11.511508 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf4b447-a823-4a1d-b9c6-6cbf90cbad73 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:11.511906 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf4b447-a823-4a1d-b9c6-6cbf90cbad73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:11.521114 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf4b447-a823-4a1d-b9c6-6cbf90cbad73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:11.522593 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf4b447-a823-4a1d-b9c6-6cbf90cbad73 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:11.533590 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 918/1818] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:04:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:04:11.683066 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:04:12.110596 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07561ece-d45d-4906-9f76-f054d2b705c3 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:12.112326 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-07561ece-d45d-4906-9f76-f054d2b705c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:12.124330 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf4b447-a823-4a1d-b9c6-6cbf90cbad73 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:12.127157 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-aaf4b447-a823-4a1d-b9c6-6cbf90cbad73 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:12.148568 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07561ece-d45d-4906-9f76-f054d2b705c3 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:12.157695 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c20a602-d220-469e-8de4-73f2f523690d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:12.158719 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c20a602-d220-469e-8de4-73f2f523690d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:12.159011 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c20a602-d220-469e-8de4-73f2f523690d admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:12.159288 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c20a602-d220-469e-8de4-73f2f523690d admin admin] PATH_INFO: `/v3/projects/911ab827aa504e5596c79ba278d157a8` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:12.160620 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c20a602-d220-469e-8de4-73f2f523690d admin admin] RBAC: Authorizing `identity:delete_project(project_id=911ab827aa504e5596c79ba278d157a8)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:12.163397 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c20a602-d220-469e-8de4-73f2f523690d admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:12.205725 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf4b447-a823-4a1d-b9c6-6cbf90cbad73 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:12.210682 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6721ed8-4174-47ff-978f-dd17df87ae0b tempest-VolumeQuotaClassesTest-204909168 tempest-VolumeQuotaClassesTest-204909168-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f079cda0e8db4d619a56b0a208f56bbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bf0a077d9ba436f9a74a68b57a88f61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:12.211458 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6721ed8-4174-47ff-978f-dd17df87ae0b tempest-VolumeQuotaClassesTest-204909168 tempest-VolumeQuotaClassesTest-204909168-project-admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:12.211817 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6721ed8-4174-47ff-978f-dd17df87ae0b tempest-VolumeQuotaClassesTest-204909168 tempest-VolumeQuotaClassesTest-204909168-project-admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:12.212760 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6721ed8-4174-47ff-978f-dd17df87ae0b tempest-VolumeQuotaClassesTest-204909168 tempest-VolumeQuotaClassesTest-204909168-project-admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:12.213506 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6721ed8-4174-47ff-978f-dd17df87ae0b tempest-VolumeQuotaClassesTest-204909168 tempest-VolumeQuotaClassesTest-204909168-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 13:04:12.213938 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6721ed8-4174-47ff-978f-dd17df87ae0b tempest-VolumeQuotaClassesTest-204909168 tempest-VolumeQuotaClassesTest-204909168-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:12.215697 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6721ed8-4174-47ff-978f-dd17df87ae0b tempest-VolumeQuotaClassesTest-204909168 tempest-VolumeQuotaClassesTest-204909168-project-admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:12.228175 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 919/1819] 10.222.0.27 () {58 vars in 1152 bytes} [Fri Jun 10 13:04:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 13:04:12.232325 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6721ed8-4174-47ff-978f-dd17df87ae0b tempest-VolumeQuotaClassesTest-204909168 tempest-VolumeQuotaClassesTest-204909168-project-admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:12.242197 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6721ed8-4174-47ff-978f-dd17df87ae0b tempest-VolumeQuotaClassesTest-204909168 tempest-VolumeQuotaClassesTest-204909168-project-admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:12.247765 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a84436-f1e6-446f-8e37-fc79485f6131 tempest-VolumeQuotaClassesTest-204909168 tempest-VolumeQuotaClassesTest-204909168-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f079cda0e8db4d619a56b0a208f56bbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bf0a077d9ba436f9a74a68b57a88f61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:12.248465 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a84436-f1e6-446f-8e37-fc79485f6131 tempest-VolumeQuotaClassesTest-204909168 tempest-VolumeQuotaClassesTest-204909168-project-admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:12.248758 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a84436-f1e6-446f-8e37-fc79485f6131 tempest-VolumeQuotaClassesTest-204909168 tempest-VolumeQuotaClassesTest-204909168-project-admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:12.249045 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a84436-f1e6-446f-8e37-fc79485f6131 tempest-VolumeQuotaClassesTest-204909168 tempest-VolumeQuotaClassesTest-204909168-project-admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:12.249735 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a84436-f1e6-446f-8e37-fc79485f6131 tempest-VolumeQuotaClassesTest-204909168 tempest-VolumeQuotaClassesTest-204909168-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:12.251662 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a84436-f1e6-446f-8e37-fc79485f6131 tempest-VolumeQuotaClassesTest-204909168 tempest-VolumeQuotaClassesTest-204909168-project-admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:12.284745 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6c20a602-d220-469e-8de4-73f2f523690d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '911ab827aa504e5596c79ba278d157a8', 'request_id': 'req-6c20a602-d220-469e-8de4-73f2f523690d'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:12.290388 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6c20a602-d220-469e-8de4-73f2f523690d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '911ab827aa504e5596c79ba278d157a8', 'request_id': 'req-6c20a602-d220-469e-8de4-73f2f523690d'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:12.297127 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6c20a602-d220-469e-8de4-73f2f523690d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '911ab827aa504e5596c79ba278d157a8', 'request_id': 'req-6c20a602-d220-469e-8de4-73f2f523690d'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:12.348276 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 920/1820] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:04:12 2022] POST /identity/v3/projects => generated 354 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 10 13:04:12.352168 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a84436-f1e6-446f-8e37-fc79485f6131 tempest-VolumeQuotaClassesTest-204909168 tempest-VolumeQuotaClassesTest-204909168-project-admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:12.377776 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6c20a602-d220-469e-8de4-73f2f523690d admin admin] The token cache is being invalidate because project 911ab827aa504e5596c79ba278d157a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:04:12.378277 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6c20a602-d220-469e-8de4-73f2f523690d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:12.378696 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6c20a602-d220-469e-8de4-73f2f523690d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:12.380326 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 901/1821] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:04:12 2022] DELETE /identity/v3/projects/911ab827aa504e5596c79ba278d157a8 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:04:12.384708 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c20a602-d220-469e-8de4-73f2f523690d admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:12.386012 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-6c20a602-d220-469e-8de4-73f2f523690d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:12.413528 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a84436-f1e6-446f-8e37-fc79485f6131 tempest-VolumeQuotaClassesTest-204909168 tempest-VolumeQuotaClassesTest-204909168-project-admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:12.418524 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8b4252-4d9e-44a0-b8b6-3b082a80b3dd tempest-VolumeQuotaClassesTest-204909168 tempest-VolumeQuotaClassesTest-204909168-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f079cda0e8db4d619a56b0a208f56bbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bf0a077d9ba436f9a74a68b57a88f61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:12.419299 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8b4252-4d9e-44a0-b8b6-3b082a80b3dd tempest-VolumeQuotaClassesTest-204909168 tempest-VolumeQuotaClassesTest-204909168-project-admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:12.419650 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8b4252-4d9e-44a0-b8b6-3b082a80b3dd tempest-VolumeQuotaClassesTest-204909168 tempest-VolumeQuotaClassesTest-204909168-project-admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:12.420077 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8b4252-4d9e-44a0-b8b6-3b082a80b3dd tempest-VolumeQuotaClassesTest-204909168 tempest-VolumeQuotaClassesTest-204909168-project-admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:12.420813 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8b4252-4d9e-44a0-b8b6-3b082a80b3dd tempest-VolumeQuotaClassesTest-204909168 tempest-VolumeQuotaClassesTest-204909168-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 13:04:12.421160 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8b4252-4d9e-44a0-b8b6-3b082a80b3dd tempest-VolumeQuotaClassesTest-204909168 tempest-VolumeQuotaClassesTest-204909168-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:12.422761 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8b4252-4d9e-44a0-b8b6-3b082a80b3dd tempest-VolumeQuotaClassesTest-204909168 tempest-VolumeQuotaClassesTest-204909168-project-admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:12.438919 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 921/1822] 10.222.0.27 () {58 vars in 1152 bytes} [Fri Jun 10 13:04:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 13:04:12.448685 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c20a602-d220-469e-8de4-73f2f523690d admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:12.481196 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8b4252-4d9e-44a0-b8b6-3b082a80b3dd tempest-VolumeQuotaClassesTest-204909168 tempest-VolumeQuotaClassesTest-204909168-project-admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:12.482969 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9d8b4252-4d9e-44a0-b8b6-3b082a80b3dd tempest-VolumeQuotaClassesTest-204909168 tempest-VolumeQuotaClassesTest-204909168-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:12.496801 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8b4252-4d9e-44a0-b8b6-3b082a80b3dd tempest-VolumeQuotaClassesTest-204909168 tempest-VolumeQuotaClassesTest-204909168-project-admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:12.505819 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f77ef7-0c9a-4826-9667-89a2715315e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:12.506710 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f77ef7-0c9a-4826-9667-89a2715315e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:12.506997 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f77ef7-0c9a-4826-9667-89a2715315e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:12.507278 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f77ef7-0c9a-4826-9667-89a2715315e9 admin admin] PATH_INFO: `/v3/users/c2d90f26d2c348dd86c9be001a3b8603` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:12.508566 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f77ef7-0c9a-4826-9667-89a2715315e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2d90f26d2c348dd86c9be001a3b8603)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:12.511570 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f77ef7-0c9a-4826-9667-89a2715315e9 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:12.515899 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2e3259-d9a6-4082-9914-c1f23ae661b8 tempest-VolumeQuotaClassesTest-204909168 tempest-VolumeQuotaClassesTest-204909168-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f079cda0e8db4d619a56b0a208f56bbb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '9bf0a077d9ba436f9a74a68b57a88f61', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:12.516867 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2e3259-d9a6-4082-9914-c1f23ae661b8 tempest-VolumeQuotaClassesTest-204909168 tempest-VolumeQuotaClassesTest-204909168-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:12.520701 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2e3259-d9a6-4082-9914-c1f23ae661b8 tempest-VolumeQuotaClassesTest-204909168 tempest-VolumeQuotaClassesTest-204909168-project-admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:12.521010 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2e3259-d9a6-4082-9914-c1f23ae661b8 tempest-VolumeQuotaClassesTest-204909168 tempest-VolumeQuotaClassesTest-204909168-project-admin] PATH_INFO: `/v3/projects/14d06c5dcce246cd943ca78034c41cb9` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:12.525307 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2e3259-d9a6-4082-9914-c1f23ae661b8 tempest-VolumeQuotaClassesTest-204909168 tempest-VolumeQuotaClassesTest-204909168-project-admin] RBAC: Authorizing `identity:delete_project(project_id=14d06c5dcce246cd943ca78034c41cb9)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:12.527533 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2e3259-d9a6-4082-9914-c1f23ae661b8 tempest-VolumeQuotaClassesTest-204909168 tempest-VolumeQuotaClassesTest-204909168-project-admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:12.731304 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-71f77ef7-0c9a-4826-9667-89a2715315e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2d90f26d2c348dd86c9be001a3b8603'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:12.744653 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-71f77ef7-0c9a-4826-9667-89a2715315e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2d90f26d2c348dd86c9be001a3b8603'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:12.860058 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-71f77ef7-0c9a-4826-9667-89a2715315e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2d90f26d2c348dd86c9be001a3b8603'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:12.860913 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-71f77ef7-0c9a-4826-9667-89a2715315e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2d90f26d2c348dd86c9be001a3b8603'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:12.870147 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:04:12.880362 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-7e2e3259-d9a6-4082-9914-c1f23ae661b8 tempest-VolumeQuotaClassesTest-204909168 tempest-VolumeQuotaClassesTest-204909168-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14d06c5dcce246cd943ca78034c41cb9', 'request_id': 'req-7e2e3259-d9a6-4082-9914-c1f23ae661b8'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:12.885886 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-71f77ef7-0c9a-4826-9667-89a2715315e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2d90f26d2c348dd86c9be001a3b8603'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:12.893941 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-7e2e3259-d9a6-4082-9914-c1f23ae661b8 tempest-VolumeQuotaClassesTest-204909168 tempest-VolumeQuotaClassesTest-204909168-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14d06c5dcce246cd943ca78034c41cb9', 'request_id': 'req-7e2e3259-d9a6-4082-9914-c1f23ae661b8'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:12.897934 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 902/1823] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:04:12 2022] DELETE /identity/v3/users/c2d90f26d2c348dd86c9be001a3b8603 => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 10 13:04:12.903852 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-7e2e3259-d9a6-4082-9914-c1f23ae661b8 tempest-VolumeQuotaClassesTest-204909168 tempest-VolumeQuotaClassesTest-204909168-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14d06c5dcce246cd943ca78034c41cb9', 'request_id': 'req-7e2e3259-d9a6-4082-9914-c1f23ae661b8'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:12.997117 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-7e2e3259-d9a6-4082-9914-c1f23ae661b8 tempest-VolumeQuotaClassesTest-204909168 tempest-VolumeQuotaClassesTest-204909168-project-admin] The token cache is being invalidate because project 14d06c5dcce246cd943ca78034c41cb9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:04:12.997402 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-7e2e3259-d9a6-4082-9914-c1f23ae661b8 tempest-VolumeQuotaClassesTest-204909168 tempest-VolumeQuotaClassesTest-204909168-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:13.000337 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-7e2e3259-d9a6-4082-9914-c1f23ae661b8 tempest-VolumeQuotaClassesTest-204909168 tempest-VolumeQuotaClassesTest-204909168-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:13.003281 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 922/1824] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:04:12 2022] DELETE /identity/v3/projects/14d06c5dcce246cd943ca78034c41cb9 => generated 0 bytes in 522 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 10 13:04:13.703096 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f77ef7-0c9a-4826-9667-89a2715315e9 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:13.705066 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-71f77ef7-0c9a-4826-9667-89a2715315e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:13.775414 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f77ef7-0c9a-4826-9667-89a2715315e9 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:13.780201 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7dd6c0-8667-4818-84fb-51645df16423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:13.781044 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7dd6c0-8667-4818-84fb-51645df16423 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:13.781420 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7dd6c0-8667-4818-84fb-51645df16423 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:13.781709 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7dd6c0-8667-4818-84fb-51645df16423 admin admin] PATH_INFO: `/v3/projects/f20717032f92451282c865fb118ab429` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:13.782971 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7dd6c0-8667-4818-84fb-51645df16423 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f20717032f92451282c865fb118ab429)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:13.785028 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7dd6c0-8667-4818-84fb-51645df16423 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:13.803175 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2e3259-d9a6-4082-9914-c1f23ae661b8 tempest-VolumeQuotaClassesTest-204909168 tempest-VolumeQuotaClassesTest-204909168-project-admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:13.804940 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-7e2e3259-d9a6-4082-9914-c1f23ae661b8 tempest-VolumeQuotaClassesTest-204909168 tempest-VolumeQuotaClassesTest-204909168-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:13.878106 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2e3259-d9a6-4082-9914-c1f23ae661b8 tempest-VolumeQuotaClassesTest-204909168 tempest-VolumeQuotaClassesTest-204909168-project-admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:13.882594 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942bffac-7c61-4729-8b44-a9893df5c853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:13.883515 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942bffac-7c61-4729-8b44-a9893df5c853 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:13.884205 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942bffac-7c61-4729-8b44-a9893df5c853 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:13.884487 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942bffac-7c61-4729-8b44-a9893df5c853 admin admin] PATH_INFO: `/v3/users/a0ae8699d725402ba626a58f22f87f6b` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:13.888443 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942bffac-7c61-4729-8b44-a9893df5c853 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0ae8699d725402ba626a58f22f87f6b)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:13.888443 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942bffac-7c61-4729-8b44-a9893df5c853 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:13.927076 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ee7dd6c0-8667-4818-84fb-51645df16423 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f20717032f92451282c865fb118ab429', 'request_id': 'req-ee7dd6c0-8667-4818-84fb-51645df16423'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:13.932723 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ee7dd6c0-8667-4818-84fb-51645df16423 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f20717032f92451282c865fb118ab429', 'request_id': 'req-ee7dd6c0-8667-4818-84fb-51645df16423'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:13.939092 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ee7dd6c0-8667-4818-84fb-51645df16423 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f20717032f92451282c865fb118ab429', 'request_id': 'req-ee7dd6c0-8667-4818-84fb-51645df16423'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:14.027482 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ee7dd6c0-8667-4818-84fb-51645df16423 admin admin] The token cache is being invalidate because project f20717032f92451282c865fb118ab429 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:04:14.028649 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ee7dd6c0-8667-4818-84fb-51645df16423 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:14.030722 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ee7dd6c0-8667-4818-84fb-51645df16423 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:14.033385 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 903/1825] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:04:13 2022] DELETE /identity/v3/projects/f20717032f92451282c865fb118ab429 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:04:14.041075 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69470367-7e19-4f17-bb83-06df3af8c7a5 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:14.041586 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69470367-7e19-4f17-bb83-06df3af8c7a5 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:14.042016 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69470367-7e19-4f17-bb83-06df3af8c7a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:14.076753 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-69470367-7e19-4f17-bb83-06df3af8c7a5 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:04:14.078209 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-69470367-7e19-4f17-bb83-06df3af8c7a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:14.096699 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:04:14.118553 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-942bffac-7c61-4729-8b44-a9893df5c853 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0ae8699d725402ba626a58f22f87f6b'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:14.146356 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-942bffac-7c61-4729-8b44-a9893df5c853 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0ae8699d725402ba626a58f22f87f6b'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:14.179399 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 904/1826] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:04:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:04:14.188610 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4e3cb2-13d3-4825-bd1d-108bc04b1eec None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:14.189214 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4e3cb2-13d3-4825-bd1d-108bc04b1eec None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:14.189676 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4e3cb2-13d3-4825-bd1d-108bc04b1eec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:14.216963 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-8c4e3cb2-13d3-4825-bd1d-108bc04b1eec None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:04:14.218572 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-8c4e3cb2-13d3-4825-bd1d-108bc04b1eec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:14.229539 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 905/1827] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:04:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 10 13:04:14.233445 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4e3cb2-13d3-4825-bd1d-108bc04b1eec None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:14.243264 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-942bffac-7c61-4729-8b44-a9893df5c853 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0ae8699d725402ba626a58f22f87f6b'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:14.243922 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4e3cb2-13d3-4825-bd1d-108bc04b1eec None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:14.244346 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-942bffac-7c61-4729-8b44-a9893df5c853 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0ae8699d725402ba626a58f22f87f6b'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:14.249442 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e752c1bb-129b-4143-b50d-0e84b4f28b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:14.251848 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e752c1bb-129b-4143-b50d-0e84b4f28b79 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:14.251848 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e752c1bb-129b-4143-b50d-0e84b4f28b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:14.251848 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e752c1bb-129b-4143-b50d-0e84b4f28b79 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:14.252161 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e752c1bb-129b-4143-b50d-0e84b4f28b79 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 13:04:14.252633 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e752c1bb-129b-4143-b50d-0e84b4f28b79 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:14.255051 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e752c1bb-129b-4143-b50d-0e84b4f28b79 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:14.270076 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-942bffac-7c61-4729-8b44-a9893df5c853 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0ae8699d725402ba626a58f22f87f6b'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:14.277190 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 906/1828] 10.222.0.27 () {58 vars in 1152 bytes} [Fri Jun 10 13:04:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 13:04:14.278329 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e752c1bb-129b-4143-b50d-0e84b4f28b79 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:14.281067 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e752c1bb-129b-4143-b50d-0e84b4f28b79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:14.285977 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 923/1829] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:04:13 2022] DELETE /identity/v3/users/a0ae8699d725402ba626a58f22f87f6b => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 10 13:04:14.289830 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fc8e10-9ed8-424d-9cb9-a0209aa17bf1 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:14.290632 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fc8e10-9ed8-424d-9cb9-a0209aa17bf1 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:14.291090 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fc8e10-9ed8-424d-9cb9-a0209aa17bf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:14.330213 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e752c1bb-129b-4143-b50d-0e84b4f28b79 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:14.341327 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-f8fc8e10-9ed8-424d-9cb9-a0209aa17bf1 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:04:14.343223 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f8fc8e10-9ed8-424d-9cb9-a0209aa17bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:14.425636 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e9fb99-0e85-4f66-8c97-50f920b83bb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:14.426424 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e9fb99-0e85-4f66-8c97-50f920b83bb1 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:14.426719 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e9fb99-0e85-4f66-8c97-50f920b83bb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:14.427000 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e9fb99-0e85-4f66-8c97-50f920b83bb1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:14.428239 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e9fb99-0e85-4f66-8c97-50f920b83bb1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:14.430987 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e9fb99-0e85-4f66-8c97-50f920b83bb1 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:14.433699 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 924/1830] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:04:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:04:14.512536 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fc8e10-9ed8-424d-9cb9-a0209aa17bf1 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:14.518644 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f8fc8e10-9ed8-424d-9cb9-a0209aa17bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:14.528080 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 907/1831] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:04:14 2022] POST /identity/v3/projects => generated 389 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:04:14.537141 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e9fb99-0e85-4f66-8c97-50f920b83bb1 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:14.658795 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e9fb99-0e85-4f66-8c97-50f920b83bb1 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:14.667659 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8e128d-e15e-46a3-9f4e-1679b317b60a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:14.668393 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8e128d-e15e-46a3-9f4e-1679b317b60a admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:14.668826 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8e128d-e15e-46a3-9f4e-1679b317b60a admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:14.669119 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8e128d-e15e-46a3-9f4e-1679b317b60a admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:14.669838 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8e128d-e15e-46a3-9f4e-1679b317b60a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:14.673643 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8e128d-e15e-46a3-9f4e-1679b317b60a admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:14.683373 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fc8e10-9ed8-424d-9cb9-a0209aa17bf1 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:14.696246 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:04:14.742263 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce912ec0-5c2c-419d-8a71-f81a60f8c1ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:14.742950 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce912ec0-5c2c-419d-8a71-f81a60f8c1ee service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:14.743760 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce912ec0-5c2c-419d-8a71-f81a60f8c1ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:14.744075 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce912ec0-5c2c-419d-8a71-f81a60f8c1ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:14.755811 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce912ec0-5c2c-419d-8a71-f81a60f8c1ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:14.757421 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce912ec0-5c2c-419d-8a71-f81a60f8c1ee service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:14.771926 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 925/1832] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:04:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 10 13:04:14.780875 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:04:14.784753 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 908/1833] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:04:14 2022] POST /identity/v3/users => generated 295 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:04:14.788447 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce912ec0-5c2c-419d-8a71-f81a60f8c1ee service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:14.798814 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce912ec0-5c2c-419d-8a71-f81a60f8c1ee service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:14.804405 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fa2bab-78b3-4cd5-81e5-c00cb204e2be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:14.806448 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa2bab-78b3-4cd5-81e5-c00cb204e2be admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:14.806976 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa2bab-78b3-4cd5-81e5-c00cb204e2be admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:14.807465 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa2bab-78b3-4cd5-81e5-c00cb204e2be admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:14.808279 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fa2bab-78b3-4cd5-81e5-c00cb204e2be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:14.809906 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fa2bab-78b3-4cd5-81e5-c00cb204e2be admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:14.820944 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 926/1834] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:04:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 10 13:04:14.825494 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8e128d-e15e-46a3-9f4e-1679b317b60a admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:14.838727 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8e128d-e15e-46a3-9f4e-1679b317b60a admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:14.844627 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c83a77-04fe-441d-b01a-96b4a7fdfd4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:14.845308 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c83a77-04fe-441d-b01a-96b4a7fdfd4e admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:14.845597 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c83a77-04fe-441d-b01a-96b4a7fdfd4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:14.845879 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c83a77-04fe-441d-b01a-96b4a7fdfd4e admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:14.846618 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c83a77-04fe-441d-b01a-96b4a7fdfd4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:14.848291 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c83a77-04fe-441d-b01a-96b4a7fdfd4e admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:14.863045 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 909/1835] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:04:14 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:04:14.869203 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fa2bab-78b3-4cd5-81e5-c00cb204e2be admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:14.877174 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fa2bab-78b3-4cd5-81e5-c00cb204e2be admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:14.882516 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d9d4fa-1ed7-4b14-a691-babc2f21b57b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:14.883407 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d9d4fa-1ed7-4b14-a691-babc2f21b57b admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:14.883687 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d9d4fa-1ed7-4b14-a691-babc2f21b57b admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:14.884056 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d9d4fa-1ed7-4b14-a691-babc2f21b57b admin admin] PATH_INFO: `/v3/projects/0a7f04411ebd4eedb556b0d6a67894e1/users/f573ea364fdf4204a7ef4c16df25704f/roles/43106c664e1b40de984c0c62701149c8` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:14.899521 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d9d4fa-1ed7-4b14-a691-babc2f21b57b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a7f04411ebd4eedb556b0d6a67894e1, user_id=f573ea364fdf4204a7ef4c16df25704f, role_id=43106c664e1b40de984c0c62701149c8)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:14.902244 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d9d4fa-1ed7-4b14-a691-babc2f21b57b admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:14.980550 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 927/1836] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:04:14 2022] PUT /identity/v3/projects/0a7f04411ebd4eedb556b0d6a67894e1/users/f573ea364fdf4204a7ef4c16df25704f/roles/43106c664e1b40de984c0c62701149c8 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:04:14.991463 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb64d7a-0d55-4931-b5ff-dd0bc30c548e None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:14.991739 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb64d7a-0d55-4931-b5ff-dd0bc30c548e None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:14.992048 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb64d7a-0d55-4931-b5ff-dd0bc30c548e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:15.018052 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-8bb64d7a-0d55-4931-b5ff-dd0bc30c548e None None] MFA Rules not processed for user `f573ea364fdf4204a7ef4c16df25704f`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:04:15.019225 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-8bb64d7a-0d55-4931-b5ff-dd0bc30c548e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:15.084176 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 910/1837] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:04:14 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:04:15.089644 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d9d4fa-1ed7-4b14-a691-babc2f21b57b admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:15.163348 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d9d4fa-1ed7-4b14-a691-babc2f21b57b admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:15.170274 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa771af-22cb-4637-9d13-bbf9393b8455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:15.171023 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa771af-22cb-4637-9d13-bbf9393b8455 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:15.171378 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa771af-22cb-4637-9d13-bbf9393b8455 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:15.171728 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa771af-22cb-4637-9d13-bbf9393b8455 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:15.172552 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa771af-22cb-4637-9d13-bbf9393b8455 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:15.174761 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa771af-22cb-4637-9d13-bbf9393b8455 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:15.281063 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 928/1838] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:04:15 2022] POST /identity/v3/projects => generated 389 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:04:15.285151 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb64d7a-0d55-4931-b5ff-dd0bc30c548e None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:15.297612 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb64d7a-0d55-4931-b5ff-dd0bc30c548e None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:15.298286 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa771af-22cb-4637-9d13-bbf9393b8455 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:15.300087 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1aa771af-22cb-4637-9d13-bbf9393b8455 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:15.370656 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb37668-91e8-46b9-8da5-a67ba225b266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:15.371339 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb37668-91e8-46b9-8da5-a67ba225b266 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:15.371628 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb37668-91e8-46b9-8da5-a67ba225b266 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:15.371938 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb37668-91e8-46b9-8da5-a67ba225b266 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:15.372179 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa771af-22cb-4637-9d13-bbf9393b8455 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:15.372640 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb37668-91e8-46b9-8da5-a67ba225b266 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:15.374562 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb37668-91e8-46b9-8da5-a67ba225b266 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:15.377047 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5305fea1-30d9-469c-a1b9-4182964f56bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:15.377904 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5305fea1-30d9-469c-a1b9-4182964f56bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:15.378186 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5305fea1-30d9-469c-a1b9-4182964f56bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:15.378460 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5305fea1-30d9-469c-a1b9-4182964f56bf admin admin] PATH_INFO: `/v3/projects/3c027d3ab445467aa62d8083c7b3bbec` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:15.380192 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5305fea1-30d9-469c-a1b9-4182964f56bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c027d3ab445467aa62d8083c7b3bbec)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:15.382061 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5305fea1-30d9-469c-a1b9-4182964f56bf admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:15.496726 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 911/1839] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:04:15 2022] POST /identity/v3/users => generated 301 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:04:15.497771 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb37668-91e8-46b9-8da5-a67ba225b266 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:15.508373 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb37668-91e8-46b9-8da5-a67ba225b266 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:15.523383 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5305fea1-30d9-469c-a1b9-4182964f56bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c027d3ab445467aa62d8083c7b3bbec', 'request_id': 'req-5305fea1-30d9-469c-a1b9-4182964f56bf'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:15.529688 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5305fea1-30d9-469c-a1b9-4182964f56bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c027d3ab445467aa62d8083c7b3bbec', 'request_id': 'req-5305fea1-30d9-469c-a1b9-4182964f56bf'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:15.536659 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5305fea1-30d9-469c-a1b9-4182964f56bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c027d3ab445467aa62d8083c7b3bbec', 'request_id': 'req-5305fea1-30d9-469c-a1b9-4182964f56bf'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:15.568343 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba2ef1a-44fe-46ab-8dd9-bcc741219c3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:15.569012 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba2ef1a-44fe-46ab-8dd9-bcc741219c3b admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:15.569301 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba2ef1a-44fe-46ab-8dd9-bcc741219c3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:15.569572 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba2ef1a-44fe-46ab-8dd9-bcc741219c3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:15.570304 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba2ef1a-44fe-46ab-8dd9-bcc741219c3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:15.572461 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba2ef1a-44fe-46ab-8dd9-bcc741219c3b admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:15.582606 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 912/1840] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:04:15 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:04:15.586894 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba2ef1a-44fe-46ab-8dd9-bcc741219c3b admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:15.597222 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba2ef1a-44fe-46ab-8dd9-bcc741219c3b admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:15.602312 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c94d84-e8bb-4323-9aa4-1664627ad614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:15.603025 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5305fea1-30d9-469c-a1b9-4182964f56bf admin admin] The token cache is being invalidate because project 3c027d3ab445467aa62d8083c7b3bbec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:04:15.603354 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c94d84-e8bb-4323-9aa4-1664627ad614 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:15.603456 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5305fea1-30d9-469c-a1b9-4182964f56bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:15.603646 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c94d84-e8bb-4323-9aa4-1664627ad614 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:15.603981 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c94d84-e8bb-4323-9aa4-1664627ad614 admin admin] PATH_INFO: `/v3/projects/a2bdbad0160e48699c47874466cfa971/users/bf3c1742bd044f9b9a99bf12750bf542/roles/b43d5a8247e244869c2c70f986f42e8a` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:15.604412 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5305fea1-30d9-469c-a1b9-4182964f56bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:15.606763 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 929/1841] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:04:15 2022] DELETE /identity/v3/projects/3c027d3ab445467aa62d8083c7b3bbec => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 10 13:04:15.615333 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5305fea1-30d9-469c-a1b9-4182964f56bf admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:15.616762 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5305fea1-30d9-469c-a1b9-4182964f56bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:15.626922 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c94d84-e8bb-4323-9aa4-1664627ad614 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2bdbad0160e48699c47874466cfa971, user_id=bf3c1742bd044f9b9a99bf12750bf542, role_id=b43d5a8247e244869c2c70f986f42e8a)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:15.629654 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c94d84-e8bb-4323-9aa4-1664627ad614 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:15.629770 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5305fea1-30d9-469c-a1b9-4182964f56bf admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:15.634815 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3594aa53-5d49-44b1-8cbe-de68dabd01c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:15.635678 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3594aa53-5d49-44b1-8cbe-de68dabd01c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:15.635994 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3594aa53-5d49-44b1-8cbe-de68dabd01c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:15.636277 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3594aa53-5d49-44b1-8cbe-de68dabd01c9 admin admin] PATH_INFO: `/v3/users/f079cda0e8db4d619a56b0a208f56bbb` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:15.637503 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3594aa53-5d49-44b1-8cbe-de68dabd01c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f079cda0e8db4d619a56b0a208f56bbb)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:15.639403 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3594aa53-5d49-44b1-8cbe-de68dabd01c9 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:15.723018 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 913/1842] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:04:15 2022] PUT /identity/v3/projects/a2bdbad0160e48699c47874466cfa971/users/bf3c1742bd044f9b9a99bf12750bf542/roles/b43d5a8247e244869c2c70f986f42e8a => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 10 13:04:15.729784 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c94d84-e8bb-4323-9aa4-1664627ad614 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:15.731371 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d7c94d84-e8bb-4323-9aa4-1664627ad614 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:15.804659 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c94d84-e8bb-4323-9aa4-1664627ad614 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:15.810012 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed233547-699a-4e61-a396-d068faacc58e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:15.810656 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed233547-699a-4e61-a396-d068faacc58e admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:15.810943 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed233547-699a-4e61-a396-d068faacc58e admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:15.811250 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed233547-699a-4e61-a396-d068faacc58e admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:15.811992 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed233547-699a-4e61-a396-d068faacc58e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:15.813671 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed233547-699a-4e61-a396-d068faacc58e admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:15.835167 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 914/1843] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:04:15 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 10 13:04:15.839515 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed233547-699a-4e61-a396-d068faacc58e admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:15.843409 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3594aa53-5d49-44b1-8cbe-de68dabd01c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f079cda0e8db4d619a56b0a208f56bbb'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:15.858352 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed233547-699a-4e61-a396-d068faacc58e admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:15.858627 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3594aa53-5d49-44b1-8cbe-de68dabd01c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f079cda0e8db4d619a56b0a208f56bbb'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:15.868824 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5113a3-f974-4e15-9b41-d597b3df07aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:15.870758 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5113a3-f974-4e15-9b41-d597b3df07aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:15.871582 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5113a3-f974-4e15-9b41-d597b3df07aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:15.871757 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5113a3-f974-4e15-9b41-d597b3df07aa admin admin] PATH_INFO: `/v3/projects/a2bdbad0160e48699c47874466cfa971/users/bf3c1742bd044f9b9a99bf12750bf542/roles/b43d5a8247e244869c2c70f986f42e8a` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:15.874353 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5113a3-f974-4e15-9b41-d597b3df07aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2bdbad0160e48699c47874466cfa971, user_id=bf3c1742bd044f9b9a99bf12750bf542, role_id=b43d5a8247e244869c2c70f986f42e8a)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:15.877680 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5113a3-f974-4e15-9b41-d597b3df07aa admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:15.890071 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 915/1844] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:04:15 2022] PUT /identity/v3/projects/a2bdbad0160e48699c47874466cfa971/users/bf3c1742bd044f9b9a99bf12750bf542/roles/b43d5a8247e244869c2c70f986f42e8a => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:04:15.900772 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f0fe55-5e2c-4ef1-87f9-6de78533dcc1 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:15.901062 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f0fe55-5e2c-4ef1-87f9-6de78533dcc1 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:15.901340 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f0fe55-5e2c-4ef1-87f9-6de78533dcc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:15.931663 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-14f0fe55-5e2c-4ef1-87f9-6de78533dcc1 None None] MFA Rules not processed for user `bf3c1742bd044f9b9a99bf12750bf542`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:04:15.932907 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-14f0fe55-5e2c-4ef1-87f9-6de78533dcc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:15.947831 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3594aa53-5d49-44b1-8cbe-de68dabd01c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f079cda0e8db4d619a56b0a208f56bbb'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:15.949176 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3594aa53-5d49-44b1-8cbe-de68dabd01c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f079cda0e8db4d619a56b0a208f56bbb'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:15.965901 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3594aa53-5d49-44b1-8cbe-de68dabd01c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f079cda0e8db4d619a56b0a208f56bbb'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:15.976195 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 930/1845] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:04:15 2022] DELETE /identity/v3/users/f079cda0e8db4d619a56b0a208f56bbb => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 10 13:04:16.025482 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 916/1846] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:04:15 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 10 13:04:16.048737 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3594aa53-5d49-44b1-8cbe-de68dabd01c9 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:16.050019 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3594aa53-5d49-44b1-8cbe-de68dabd01c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:16.130441 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3594aa53-5d49-44b1-8cbe-de68dabd01c9 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:16.135847 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cc02f2-5654-415a-a69f-138831210353 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6cc95a563b941ac92ab283aa0d5a5d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:16.136845 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cc02f2-5654-415a-a69f-138831210353 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:16.137461 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cc02f2-5654-415a-a69f-138831210353 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:16.137592 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cc02f2-5654-415a-a69f-138831210353 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:16.202693 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cc02f2-5654-415a-a69f-138831210353 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:16.204177 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cc02f2-5654-415a-a69f-138831210353 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:16.214193 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 931/1847] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:04:16 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:04:16.369779 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f0fe55-5e2c-4ef1-87f9-6de78533dcc1 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:16.371248 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-14f0fe55-5e2c-4ef1-87f9-6de78533dcc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:16.421891 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:04:16.485961 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f0fe55-5e2c-4ef1-87f9-6de78533dcc1 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:16.548862 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193b6e9b-84e1-4bec-86b0-6a0e78a8f832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:16.549853 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193b6e9b-84e1-4bec-86b0-6a0e78a8f832 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:16.550363 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193b6e9b-84e1-4bec-86b0-6a0e78a8f832 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:16.550594 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193b6e9b-84e1-4bec-86b0-6a0e78a8f832 admin admin] PATH_INFO: `/v3/users/f573ea364fdf4204a7ef4c16df25704f` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:16.552229 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193b6e9b-84e1-4bec-86b0-6a0e78a8f832 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f573ea364fdf4204a7ef4c16df25704f)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:16.554825 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193b6e9b-84e1-4bec-86b0-6a0e78a8f832 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:16.582341 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:04:16.765247 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cc02f2-5654-415a-a69f-138831210353 service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:16.768757 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f1cc02f2-5654-415a-a69f-138831210353 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:16.806721 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cc02f2-5654-415a-a69f-138831210353 service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:16.809871 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-193b6e9b-84e1-4bec-86b0-6a0e78a8f832 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f573ea364fdf4204a7ef4c16df25704f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:16.811978 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7821a8cc-c6cd-40a4-abf6-514d5ecb333d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:16.812957 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7821a8cc-c6cd-40a4-abf6-514d5ecb333d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:16.813253 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7821a8cc-c6cd-40a4-abf6-514d5ecb333d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:16.813523 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7821a8cc-c6cd-40a4-abf6-514d5ecb333d admin admin] PATH_INFO: `/v3/projects/9bf0a077d9ba436f9a74a68b57a88f61` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:16.814716 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7821a8cc-c6cd-40a4-abf6-514d5ecb333d admin admin] RBAC: Authorizing `identity:delete_project(project_id=9bf0a077d9ba436f9a74a68b57a88f61)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:16.816707 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7821a8cc-c6cd-40a4-abf6-514d5ecb333d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:16.817415 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-193b6e9b-84e1-4bec-86b0-6a0e78a8f832 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f573ea364fdf4204a7ef4c16df25704f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:16.899441 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-193b6e9b-84e1-4bec-86b0-6a0e78a8f832 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f573ea364fdf4204a7ef4c16df25704f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:16.900079 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-193b6e9b-84e1-4bec-86b0-6a0e78a8f832 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f573ea364fdf4204a7ef4c16df25704f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:16.912229 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-193b6e9b-84e1-4bec-86b0-6a0e78a8f832 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f573ea364fdf4204a7ef4c16df25704f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:16.918647 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 917/1848] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:04:16 2022] DELETE /identity/v3/users/f573ea364fdf4204a7ef4c16df25704f => generated 0 bytes in 549 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:04:16.922847 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d0f4fe-28b7-49a2-b1dc-fe2b0402a433 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:16.923138 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d0f4fe-28b7-49a2-b1dc-fe2b0402a433 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:16.923413 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d0f4fe-28b7-49a2-b1dc-fe2b0402a433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:16.940999 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-78d0f4fe-28b7-49a2-b1dc-fe2b0402a433 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:04:16.941819 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-7821a8cc-c6cd-40a4-abf6-514d5ecb333d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9bf0a077d9ba436f9a74a68b57a88f61', 'request_id': 'req-7821a8cc-c6cd-40a4-abf6-514d5ecb333d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:16.942280 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-78d0f4fe-28b7-49a2-b1dc-fe2b0402a433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:16.947927 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-7821a8cc-c6cd-40a4-abf6-514d5ecb333d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9bf0a077d9ba436f9a74a68b57a88f61', 'request_id': 'req-7821a8cc-c6cd-40a4-abf6-514d5ecb333d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:16.954030 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-7821a8cc-c6cd-40a4-abf6-514d5ecb333d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9bf0a077d9ba436f9a74a68b57a88f61', 'request_id': 'req-7821a8cc-c6cd-40a4-abf6-514d5ecb333d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:17.000167 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 918/1849] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:04:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 10 13:04:17.010378 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d0f4fe-28b7-49a2-b1dc-fe2b0402a433 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:17.011877 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-78d0f4fe-28b7-49a2-b1dc-fe2b0402a433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:17.032528 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-7821a8cc-c6cd-40a4-abf6-514d5ecb333d admin admin] The token cache is being invalidate because project 9bf0a077d9ba436f9a74a68b57a88f61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:04:17.032829 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-7821a8cc-c6cd-40a4-abf6-514d5ecb333d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:17.033465 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-7821a8cc-c6cd-40a4-abf6-514d5ecb333d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:17.035277 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 932/1850] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:04:16 2022] DELETE /identity/v3/projects/9bf0a077d9ba436f9a74a68b57a88f61 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:04:17.044591 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067a9e99-65fd-4756-a15f-757b0fa9d227 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:17.045123 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067a9e99-65fd-4756-a15f-757b0fa9d227 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:17.045402 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067a9e99-65fd-4756-a15f-757b0fa9d227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:17.065053 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-067a9e99-65fd-4756-a15f-757b0fa9d227 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:04:17.066291 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-067a9e99-65fd-4756-a15f-757b0fa9d227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:17.075030 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 933/1851] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:04:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 10 13:04:17.075845 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d0f4fe-28b7-49a2-b1dc-fe2b0402a433 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:17.087674 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ca456c-9496-4151-bb6c-668474da2f05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:17.088366 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ca456c-9496-4151-bb6c-668474da2f05 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:17.088657 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ca456c-9496-4151-bb6c-668474da2f05 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:17.088992 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ca456c-9496-4151-bb6c-668474da2f05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:17.090806 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a3ca456c-9496-4151-bb6c-668474da2f05 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:17.104010 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ca456c-9496-4151-bb6c-668474da2f05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:17.104778 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8de74c-f700-48a7-8275-25a5625dd9a1 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:17.105074 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8de74c-f700-48a7-8275-25a5625dd9a1 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:17.105354 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8de74c-f700-48a7-8275-25a5625dd9a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:17.105632 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ca456c-9496-4151-bb6c-668474da2f05 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:17.117914 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 919/1852] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:04:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:04:17.124786 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-8e8de74c-f700-48a7-8275-25a5625dd9a1 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:04:17.126175 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8e8de74c-f700-48a7-8275-25a5625dd9a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:17.137120 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 934/1853] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:04:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 10 13:04:17.142785 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ca456c-9496-4151-bb6c-668474da2f05 service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:17.149262 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ca456c-9496-4151-bb6c-668474da2f05 service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:17.154025 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00b6a5a-b472-410d-972c-dbeb2cd9e896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:17.154730 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00b6a5a-b472-410d-972c-dbeb2cd9e896 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:17.155016 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00b6a5a-b472-410d-972c-dbeb2cd9e896 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:17.155295 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00b6a5a-b472-410d-972c-dbeb2cd9e896 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:17.156191 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00b6a5a-b472-410d-972c-dbeb2cd9e896 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 13:04:17.156505 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00b6a5a-b472-410d-972c-dbeb2cd9e896 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:17.158097 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00b6a5a-b472-410d-972c-dbeb2cd9e896 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:17.170746 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 920/1854] 10.222.0.27 () {58 vars in 1152 bytes} [Fri Jun 10 13:04:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 13:04:17.175258 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8de74c-f700-48a7-8275-25a5625dd9a1 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:17.185732 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8de74c-f700-48a7-8275-25a5625dd9a1 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:17.190753 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7140fb9a-6649-40c7-b9a5-0b02f4cd1d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:17.191417 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7140fb9a-6649-40c7-b9a5-0b02f4cd1d1f admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:17.191707 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7140fb9a-6649-40c7-b9a5-0b02f4cd1d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:17.192031 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7140fb9a-6649-40c7-b9a5-0b02f4cd1d1f admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:17.192892 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7140fb9a-6649-40c7-b9a5-0b02f4cd1d1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:17.196448 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7140fb9a-6649-40c7-b9a5-0b02f4cd1d1f admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:17.301706 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 935/1855] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:04:17 2022] POST /identity/v3/projects => generated 387 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:04:17.306598 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00b6a5a-b472-410d-972c-dbeb2cd9e896 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:17.318012 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00b6a5a-b472-410d-972c-dbeb2cd9e896 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:17.388194 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd477e1-b6a8-4d76-b608-36cec7a7f5d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:17.388880 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd477e1-b6a8-4d76-b608-36cec7a7f5d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:17.389167 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd477e1-b6a8-4d76-b608-36cec7a7f5d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:17.389449 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd477e1-b6a8-4d76-b608-36cec7a7f5d5 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:17.390154 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd477e1-b6a8-4d76-b608-36cec7a7f5d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:17.392160 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd477e1-b6a8-4d76-b608-36cec7a7f5d5 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:17.463350 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 921/1856] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:04:17 2022] POST /identity/v3/users => generated 294 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:04:17.467436 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7140fb9a-6649-40c7-b9a5-0b02f4cd1d1f admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:17.486908 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7140fb9a-6649-40c7-b9a5-0b02f4cd1d1f admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:17.492948 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85674f64-65b0-42a5-aded-726f68800fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:17.493811 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85674f64-65b0-42a5-aded-726f68800fe8 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:17.494097 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85674f64-65b0-42a5-aded-726f68800fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:17.494377 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85674f64-65b0-42a5-aded-726f68800fe8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:17.495114 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85674f64-65b0-42a5-aded-726f68800fe8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:17.496916 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85674f64-65b0-42a5-aded-726f68800fe8 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:17.509265 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 936/1857] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:04:17 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:04:17.517197 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd477e1-b6a8-4d76-b608-36cec7a7f5d5 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:17.528447 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd477e1-b6a8-4d76-b608-36cec7a7f5d5 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:17.533589 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca2500d-18cb-4ea5-beee-7a68ba03f2d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:17.534411 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca2500d-18cb-4ea5-beee-7a68ba03f2d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:17.534701 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca2500d-18cb-4ea5-beee-7a68ba03f2d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:17.534986 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca2500d-18cb-4ea5-beee-7a68ba03f2d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:17.535712 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca2500d-18cb-4ea5-beee-7a68ba03f2d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:17.537337 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca2500d-18cb-4ea5-beee-7a68ba03f2d2 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:17.549716 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 922/1858] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:04:17 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:04:17.554087 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85674f64-65b0-42a5-aded-726f68800fe8 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:17.564530 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85674f64-65b0-42a5-aded-726f68800fe8 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:17.569467 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370d142e-4a40-4317-8840-b3d44376da62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:17.571616 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370d142e-4a40-4317-8840-b3d44376da62 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:17.571616 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370d142e-4a40-4317-8840-b3d44376da62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:17.571616 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370d142e-4a40-4317-8840-b3d44376da62 admin admin] PATH_INFO: `/v3/projects/be33831f5c2d408f9d948d6f7ebbf674/users/042f62a7fbf347f4b822f1edd5389d26/roles/43106c664e1b40de984c0c62701149c8` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:17.586187 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370d142e-4a40-4317-8840-b3d44376da62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be33831f5c2d408f9d948d6f7ebbf674, user_id=042f62a7fbf347f4b822f1edd5389d26, role_id=43106c664e1b40de984c0c62701149c8)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:17.590058 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370d142e-4a40-4317-8840-b3d44376da62 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:17.640933 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca2500d-18cb-4ea5-beee-7a68ba03f2d2 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:17.640933 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-aca2500d-18cb-4ea5-beee-7a68ba03f2d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:17.654582 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca2500d-18cb-4ea5-beee-7a68ba03f2d2 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:17.659725 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fab2042-c819-4a8e-bfee-5085569bae50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:17.660648 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fab2042-c819-4a8e-bfee-5085569bae50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:17.660961 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fab2042-c819-4a8e-bfee-5085569bae50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:17.661237 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fab2042-c819-4a8e-bfee-5085569bae50 admin admin] PATH_INFO: `/v3/projects/0a7f04411ebd4eedb556b0d6a67894e1` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:17.662436 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fab2042-c819-4a8e-bfee-5085569bae50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a7f04411ebd4eedb556b0d6a67894e1)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:17.664435 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fab2042-c819-4a8e-bfee-5085569bae50 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:17.700139 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 937/1859] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:04:17 2022] PUT /identity/v3/projects/be33831f5c2d408f9d948d6f7ebbf674/users/042f62a7fbf347f4b822f1edd5389d26/roles/43106c664e1b40de984c0c62701149c8 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:04:17.708014 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254054a2-64ec-41f7-aff2-26e3944321c6 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:17.708449 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254054a2-64ec-41f7-aff2-26e3944321c6 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:17.708733 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254054a2-64ec-41f7-aff2-26e3944321c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:17.740933 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-254054a2-64ec-41f7-aff2-26e3944321c6 None None] MFA Rules not processed for user `042f62a7fbf347f4b822f1edd5389d26`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:04:17.742553 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-254054a2-64ec-41f7-aff2-26e3944321c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:17.834754 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2fab2042-c819-4a8e-bfee-5085569bae50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a7f04411ebd4eedb556b0d6a67894e1', 'request_id': 'req-2fab2042-c819-4a8e-bfee-5085569bae50'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:17.856178 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2fab2042-c819-4a8e-bfee-5085569bae50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a7f04411ebd4eedb556b0d6a67894e1', 'request_id': 'req-2fab2042-c819-4a8e-bfee-5085569bae50'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:17.856281 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 938/1860] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:04:17 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 10 13:04:17.856281 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254054a2-64ec-41f7-aff2-26e3944321c6 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:17.864692 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2fab2042-c819-4a8e-bfee-5085569bae50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a7f04411ebd4eedb556b0d6a67894e1', 'request_id': 'req-2fab2042-c819-4a8e-bfee-5085569bae50'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:17.934296 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2fab2042-c819-4a8e-bfee-5085569bae50 admin admin] The token cache is being invalidate because project 0a7f04411ebd4eedb556b0d6a67894e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:04:17.934678 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2fab2042-c819-4a8e-bfee-5085569bae50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:17.935360 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2fab2042-c819-4a8e-bfee-5085569bae50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:17.939992 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 923/1861] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:04:17 2022] DELETE /identity/v3/projects/0a7f04411ebd4eedb556b0d6a67894e1 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 10 13:04:17.941127 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fab2042-c819-4a8e-bfee-5085569bae50 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:17.942777 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-2fab2042-c819-4a8e-bfee-5085569bae50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:17.947406 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254054a2-64ec-41f7-aff2-26e3944321c6 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:17.967561 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:04:18.019378 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fab2042-c819-4a8e-bfee-5085569bae50 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:18.019490 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa745f90-deeb-4bb1-aee3-0affaaee5cbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:18.020204 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa745f90-deeb-4bb1-aee3-0affaaee5cbe admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:18.020622 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa745f90-deeb-4bb1-aee3-0affaaee5cbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:18.021000 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa745f90-deeb-4bb1-aee3-0affaaee5cbe admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:18.021878 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa745f90-deeb-4bb1-aee3-0affaaee5cbe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:18.025095 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa745f90-deeb-4bb1-aee3-0affaaee5cbe admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:18.025449 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792fc83f-27f1-4094-841a-1ffd9022964a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:18.026684 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792fc83f-27f1-4094-841a-1ffd9022964a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:18.027084 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792fc83f-27f1-4094-841a-1ffd9022964a admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:18.027546 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792fc83f-27f1-4094-841a-1ffd9022964a admin admin] PATH_INFO: `/v3/users/bf3c1742bd044f9b9a99bf12750bf542` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:18.029005 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792fc83f-27f1-4094-841a-1ffd9022964a admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf3c1742bd044f9b9a99bf12750bf542)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:18.031350 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792fc83f-27f1-4094-841a-1ffd9022964a admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:18.061808 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:04:18.146053 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 939/1862] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:04:17 2022] POST /identity/v3/projects => generated 387 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 10 13:04:18.148144 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa745f90-deeb-4bb1-aee3-0affaaee5cbe admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:18.150909 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-aa745f90-deeb-4bb1-aee3-0affaaee5cbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:18.214523 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa745f90-deeb-4bb1-aee3-0affaaee5cbe admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:18.219822 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ff746b-a78c-4d99-9584-89277b954fb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:18.220556 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ff746b-a78c-4d99-9584-89277b954fb7 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:18.220888 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ff746b-a78c-4d99-9584-89277b954fb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:18.221167 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ff746b-a78c-4d99-9584-89277b954fb7 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:18.221868 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ff746b-a78c-4d99-9584-89277b954fb7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:18.223761 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ff746b-a78c-4d99-9584-89277b954fb7 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:18.256802 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-792fc83f-27f1-4094-841a-1ffd9022964a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf3c1742bd044f9b9a99bf12750bf542'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:18.268422 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-792fc83f-27f1-4094-841a-1ffd9022964a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf3c1742bd044f9b9a99bf12750bf542'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:18.295484 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 940/1863] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:04:18 2022] POST /identity/v3/users => generated 294 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:04:18.300405 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ff746b-a78c-4d99-9584-89277b954fb7 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:18.311016 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ff746b-a78c-4d99-9584-89277b954fb7 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:18.316256 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c79f275-f57a-403d-8c4c-3a85ee35a080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:18.316910 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c79f275-f57a-403d-8c4c-3a85ee35a080 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:18.317194 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c79f275-f57a-403d-8c4c-3a85ee35a080 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:18.317475 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c79f275-f57a-403d-8c4c-3a85ee35a080 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:18.318225 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c79f275-f57a-403d-8c4c-3a85ee35a080 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:18.319776 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c79f275-f57a-403d-8c4c-3a85ee35a080 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:18.330952 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 941/1864] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:04:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 10 13:04:18.336148 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c79f275-f57a-403d-8c4c-3a85ee35a080 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:18.346743 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c79f275-f57a-403d-8c4c-3a85ee35a080 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:18.351911 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67218ccb-e8a3-4b7d-a84e-e3e811581090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:18.352732 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67218ccb-e8a3-4b7d-a84e-e3e811581090 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:18.353089 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67218ccb-e8a3-4b7d-a84e-e3e811581090 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:18.353448 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67218ccb-e8a3-4b7d-a84e-e3e811581090 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:18.354237 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67218ccb-e8a3-4b7d-a84e-e3e811581090 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:18.355977 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67218ccb-e8a3-4b7d-a84e-e3e811581090 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:18.362501 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-792fc83f-27f1-4094-841a-1ffd9022964a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf3c1742bd044f9b9a99bf12750bf542'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:18.364072 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-792fc83f-27f1-4094-841a-1ffd9022964a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf3c1742bd044f9b9a99bf12750bf542'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:18.368642 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 942/1865] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:04:18 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 10 13:04:18.374369 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67218ccb-e8a3-4b7d-a84e-e3e811581090 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:18.383424 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-792fc83f-27f1-4094-841a-1ffd9022964a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf3c1742bd044f9b9a99bf12750bf542'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:18.386535 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67218ccb-e8a3-4b7d-a84e-e3e811581090 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:18.391612 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 924/1866] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:04:17 2022] DELETE /identity/v3/users/bf3c1742bd044f9b9a99bf12750bf542 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 10 13:04:18.393379 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e0b977-2cf7-4d00-82c6-d5dee2bcec18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:18.394440 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e0b977-2cf7-4d00-82c6-d5dee2bcec18 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:18.394765 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e0b977-2cf7-4d00-82c6-d5dee2bcec18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:18.395493 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e0b977-2cf7-4d00-82c6-d5dee2bcec18 admin admin] PATH_INFO: `/v3/projects/2e78d3d80a334b2294de1d0d6bbfbc02/users/361442add93246b2bedbec81ab045663/roles/43106c664e1b40de984c0c62701149c8` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:18.434435 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e0b977-2cf7-4d00-82c6-d5dee2bcec18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e78d3d80a334b2294de1d0d6bbfbc02, user_id=361442add93246b2bedbec81ab045663, role_id=43106c664e1b40de984c0c62701149c8)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:18.438324 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e0b977-2cf7-4d00-82c6-d5dee2bcec18 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:18.513043 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:04:18.584260 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 943/1867] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:04:18 2022] PUT /identity/v3/projects/2e78d3d80a334b2294de1d0d6bbfbc02/users/361442add93246b2bedbec81ab045663/roles/43106c664e1b40de984c0c62701149c8 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:04:18.592091 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6e2b43-30d5-4d4f-bf6b-082317b89ca3 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:18.592376 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6e2b43-30d5-4d4f-bf6b-082317b89ca3 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:18.593175 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6e2b43-30d5-4d4f-bf6b-082317b89ca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:18.635887 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-2c6e2b43-30d5-4d4f-bf6b-082317b89ca3 None None] MFA Rules not processed for user `361442add93246b2bedbec81ab045663`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:04:18.637258 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-2c6e2b43-30d5-4d4f-bf6b-082317b89ca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:18.678760 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:04:18.714261 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 925/1868] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:04:18 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:04:18.718511 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e0b977-2cf7-4d00-82c6-d5dee2bcec18 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:18.723118 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f4e0b977-2cf7-4d00-82c6-d5dee2bcec18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:18.810866 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e0b977-2cf7-4d00-82c6-d5dee2bcec18 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:18.815961 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b0dfb4-a15b-42a1-8f87-120848b8b385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:18.816635 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b0dfb4-a15b-42a1-8f87-120848b8b385 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:18.816967 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b0dfb4-a15b-42a1-8f87-120848b8b385 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:18.817247 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b0dfb4-a15b-42a1-8f87-120848b8b385 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:18.817945 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b0dfb4-a15b-42a1-8f87-120848b8b385 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:18.819968 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b0dfb4-a15b-42a1-8f87-120848b8b385 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:18.938954 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 944/1869] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:04:18 2022] POST /identity/v3/projects => generated 387 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:04:18.943156 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6e2b43-30d5-4d4f-bf6b-082317b89ca3 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:18.956839 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6e2b43-30d5-4d4f-bf6b-082317b89ca3 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:19.023006 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:04:19.027201 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ffad87-0389-4504-aefe-8210ec80e911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:19.027201 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ffad87-0389-4504-aefe-8210ec80e911 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:19.027201 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ffad87-0389-4504-aefe-8210ec80e911 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:19.027201 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ffad87-0389-4504-aefe-8210ec80e911 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:19.029301 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ffad87-0389-4504-aefe-8210ec80e911 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:19.031896 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ffad87-0389-4504-aefe-8210ec80e911 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:19.033068 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b0dfb4-a15b-42a1-8f87-120848b8b385 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:19.040459 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-29b0dfb4-a15b-42a1-8f87-120848b8b385 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:19.065369 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b0dfb4-a15b-42a1-8f87-120848b8b385 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:19.078632 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837b18a5-8dc1-4e09-ae44-b2c22cf4b10f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:19.079575 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837b18a5-8dc1-4e09-ae44-b2c22cf4b10f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:19.079892 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837b18a5-8dc1-4e09-ae44-b2c22cf4b10f admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:19.080151 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837b18a5-8dc1-4e09-ae44-b2c22cf4b10f admin admin] PATH_INFO: `/v3/projects/a2bdbad0160e48699c47874466cfa971` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:19.081574 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837b18a5-8dc1-4e09-ae44-b2c22cf4b10f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2bdbad0160e48699c47874466cfa971)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:19.083484 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837b18a5-8dc1-4e09-ae44-b2c22cf4b10f admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:19.108530 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:04:19.124352 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 926/1870] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:04:18 2022] POST /identity/v3/users => generated 300 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 10 13:04:19.128883 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ffad87-0389-4504-aefe-8210ec80e911 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:19.145711 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ffad87-0389-4504-aefe-8210ec80e911 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:19.190781 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ff4c14-4a7c-4c60-8732-250e96e1f502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:19.190781 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ff4c14-4a7c-4c60-8732-250e96e1f502 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:19.190781 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ff4c14-4a7c-4c60-8732-250e96e1f502 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:19.190781 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ff4c14-4a7c-4c60-8732-250e96e1f502 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:19.191379 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ff4c14-4a7c-4c60-8732-250e96e1f502 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:19.191379 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ff4c14-4a7c-4c60-8732-250e96e1f502 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:19.191379 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 927/1871] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:04:19 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:04:19.191379 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ff4c14-4a7c-4c60-8732-250e96e1f502 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:19.192785 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ff4c14-4a7c-4c60-8732-250e96e1f502 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:19.198789 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1660537-4ac6-4f65-ab59-70323c9ac5f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:19.199689 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1660537-4ac6-4f65-ab59-70323c9ac5f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:19.200010 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1660537-4ac6-4f65-ab59-70323c9ac5f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:19.200293 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1660537-4ac6-4f65-ab59-70323c9ac5f4 admin admin] PATH_INFO: `/v3/projects/946f4e6d0aa34c31a13a2f7a5e2e6d07/users/d81544b2fb0d47509711754805d489f7/roles/b43d5a8247e244869c2c70f986f42e8a` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:19.217593 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1660537-4ac6-4f65-ab59-70323c9ac5f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=946f4e6d0aa34c31a13a2f7a5e2e6d07, user_id=d81544b2fb0d47509711754805d489f7, role_id=b43d5a8247e244869c2c70f986f42e8a)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:19.220376 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1660537-4ac6-4f65-ab59-70323c9ac5f4 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:19.594271 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 928/1872] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:04:19 2022] PUT /identity/v3/projects/946f4e6d0aa34c31a13a2f7a5e2e6d07/users/d81544b2fb0d47509711754805d489f7/roles/b43d5a8247e244869c2c70f986f42e8a => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 10 13:04:19.594271 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1660537-4ac6-4f65-ab59-70323c9ac5f4 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:19.594485 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-837b18a5-8dc1-4e09-ae44-b2c22cf4b10f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2bdbad0160e48699c47874466cfa971', 'request_id': 'req-837b18a5-8dc1-4e09-ae44-b2c22cf4b10f'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:19.594485 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-837b18a5-8dc1-4e09-ae44-b2c22cf4b10f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2bdbad0160e48699c47874466cfa971', 'request_id': 'req-837b18a5-8dc1-4e09-ae44-b2c22cf4b10f'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:19.600790 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-837b18a5-8dc1-4e09-ae44-b2c22cf4b10f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2bdbad0160e48699c47874466cfa971', 'request_id': 'req-837b18a5-8dc1-4e09-ae44-b2c22cf4b10f'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:19.645038 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1660537-4ac6-4f65-ab59-70323c9ac5f4 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:19.699756 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4a8036-7e46-4a47-bf39-730bc04f82c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:19.700470 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4a8036-7e46-4a47-bf39-730bc04f82c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:19.700758 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4a8036-7e46-4a47-bf39-730bc04f82c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:19.701036 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4a8036-7e46-4a47-bf39-730bc04f82c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:19.701770 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4a8036-7e46-4a47-bf39-730bc04f82c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:19.703368 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4a8036-7e46-4a47-bf39-730bc04f82c8 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:19.713854 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 929/1873] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:04:19 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 10 13:04:19.718263 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4a8036-7e46-4a47-bf39-730bc04f82c8 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:19.722483 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-837b18a5-8dc1-4e09-ae44-b2c22cf4b10f admin admin] The token cache is being invalidate because project a2bdbad0160e48699c47874466cfa971 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:04:19.722807 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-837b18a5-8dc1-4e09-ae44-b2c22cf4b10f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:19.723600 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-837b18a5-8dc1-4e09-ae44-b2c22cf4b10f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:19.725435 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 945/1874] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:04:19 2022] DELETE /identity/v3/projects/a2bdbad0160e48699c47874466cfa971 => generated 0 bytes in 693 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:04:19.730366 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bfb217-add6-4d59-99c2-d3d56a726510 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:19.730659 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bfb217-add6-4d59-99c2-d3d56a726510 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:19.730943 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bfb217-add6-4d59-99c2-d3d56a726510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:19.734750 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4a8036-7e46-4a47-bf39-730bc04f82c8 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:19.740404 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9e105a-88a4-4a12-86a6-35663fdfddf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:19.741625 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9e105a-88a4-4a12-86a6-35663fdfddf9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:19.742140 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9e105a-88a4-4a12-86a6-35663fdfddf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:19.742610 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9e105a-88a4-4a12-86a6-35663fdfddf9 admin admin] PATH_INFO: `/v3/projects/946f4e6d0aa34c31a13a2f7a5e2e6d07/users/d81544b2fb0d47509711754805d489f7/roles/b43d5a8247e244869c2c70f986f42e8a` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:19.745642 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9e105a-88a4-4a12-86a6-35663fdfddf9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=946f4e6d0aa34c31a13a2f7a5e2e6d07, user_id=d81544b2fb0d47509711754805d489f7, role_id=b43d5a8247e244869c2c70f986f42e8a)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:19.748550 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9e105a-88a4-4a12-86a6-35663fdfddf9 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:19.750840 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-c9bfb217-add6-4d59-99c2-d3d56a726510 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:04:19.752152 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c9bfb217-add6-4d59-99c2-d3d56a726510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:19.760293 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 930/1875] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:04:19 2022] PUT /identity/v3/projects/946f4e6d0aa34c31a13a2f7a5e2e6d07/users/d81544b2fb0d47509711754805d489f7/roles/b43d5a8247e244869c2c70f986f42e8a => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 10 13:04:19.770195 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3af6289-371a-4a16-bbaf-ff8be2c7e174 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:19.770486 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3af6289-371a-4a16-bbaf-ff8be2c7e174 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:19.770764 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3af6289-371a-4a16-bbaf-ff8be2c7e174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:19.799894 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-b3af6289-371a-4a16-bbaf-ff8be2c7e174 None None] MFA Rules not processed for user `d81544b2fb0d47509711754805d489f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:04:19.804323 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b3af6289-371a-4a16-bbaf-ff8be2c7e174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:19.825524 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 946/1876] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:04:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:04:19.839502 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b549fff-5b62-42a5-8ff5-10364d93bbaa None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:19.839792 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b549fff-5b62-42a5-8ff5-10364d93bbaa None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:19.840105 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b549fff-5b62-42a5-8ff5-10364d93bbaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:19.883177 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-4b549fff-5b62-42a5-8ff5-10364d93bbaa None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:04:19.888748 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-4b549fff-5b62-42a5-8ff5-10364d93bbaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:19.901857 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 947/1877] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:04:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:04:19.905859 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b549fff-5b62-42a5-8ff5-10364d93bbaa None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:19.918997 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b549fff-5b62-42a5-8ff5-10364d93bbaa None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:19.922926 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272bd02f-0922-494b-ac9e-bd85b81c1fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:19.923813 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272bd02f-0922-494b-ac9e-bd85b81c1fc5 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:19.924198 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272bd02f-0922-494b-ac9e-bd85b81c1fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:19.924548 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272bd02f-0922-494b-ac9e-bd85b81c1fc5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:19.925367 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272bd02f-0922-494b-ac9e-bd85b81c1fc5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 13:04:19.925659 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 931/1878] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:04:19 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 10 13:04:19.928331 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272bd02f-0922-494b-ac9e-bd85b81c1fc5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:19.930122 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272bd02f-0922-494b-ac9e-bd85b81c1fc5 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:19.937657 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3af6289-371a-4a16-bbaf-ff8be2c7e174 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:19.939087 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b3af6289-371a-4a16-bbaf-ff8be2c7e174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:19.951290 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 948/1879] 10.222.0.27 () {58 vars in 1152 bytes} [Fri Jun 10 13:04:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 13:04:19.956080 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272bd02f-0922-494b-ac9e-bd85b81c1fc5 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:19.967299 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272bd02f-0922-494b-ac9e-bd85b81c1fc5 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:19.973242 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97508d3a-f657-423c-87f9-817490405ca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:19.979006 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97508d3a-f657-423c-87f9-817490405ca2 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:19.979006 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97508d3a-f657-423c-87f9-817490405ca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:19.979006 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97508d3a-f657-423c-87f9-817490405ca2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:19.979006 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97508d3a-f657-423c-87f9-817490405ca2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:19.979006 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97508d3a-f657-423c-87f9-817490405ca2 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:20.022130 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3af6289-371a-4a16-bbaf-ff8be2c7e174 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:20.026316 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24078368-3161-4325-93c9-fdcccb885fd7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6cc95a563b941ac92ab283aa0d5a5d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:20.027006 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24078368-3161-4325-93c9-fdcccb885fd7 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:20.027294 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24078368-3161-4325-93c9-fdcccb885fd7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:20.027574 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24078368-3161-4325-93c9-fdcccb885fd7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:20.032211 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:04:20.039616 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24078368-3161-4325-93c9-fdcccb885fd7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:20.041186 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24078368-3161-4325-93c9-fdcccb885fd7 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:20.051780 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 932/1880] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:04:19 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 10 13:04:20.099848 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 949/1881] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:04:19 2022] POST /identity/v3/projects => generated 381 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 10 13:04:20.104532 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24078368-3161-4325-93c9-fdcccb885fd7 service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:20.116693 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24078368-3161-4325-93c9-fdcccb885fd7 service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:20.186865 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337b273c-1d08-4c42-af58-ad5d7e57a5c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:20.187557 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337b273c-1d08-4c42-af58-ad5d7e57a5c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:20.187848 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337b273c-1d08-4c42-af58-ad5d7e57a5c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:20.188152 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337b273c-1d08-4c42-af58-ad5d7e57a5c9 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:20.188850 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337b273c-1d08-4c42-af58-ad5d7e57a5c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:20.190820 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337b273c-1d08-4c42-af58-ad5d7e57a5c9 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:20.356695 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 933/1882] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:04:20 2022] POST /identity/v3/users => generated 291 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:04:20.360393 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97508d3a-f657-423c-87f9-817490405ca2 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:20.375763 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97508d3a-f657-423c-87f9-817490405ca2 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:20.382560 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a518f176-50df-429d-bfb2-b50472bb1817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:20.383248 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a518f176-50df-429d-bfb2-b50472bb1817 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:20.383536 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a518f176-50df-429d-bfb2-b50472bb1817 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:20.384062 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a518f176-50df-429d-bfb2-b50472bb1817 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:20.384800 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a518f176-50df-429d-bfb2-b50472bb1817 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:20.386457 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a518f176-50df-429d-bfb2-b50472bb1817 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:20.402520 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 950/1883] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:04:20 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:04:20.404943 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337b273c-1d08-4c42-af58-ad5d7e57a5c9 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:20.417229 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337b273c-1d08-4c42-af58-ad5d7e57a5c9 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:20.422202 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca0ee4b-fd32-4bad-8dde-53fbac5d9b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:20.425267 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca0ee4b-fd32-4bad-8dde-53fbac5d9b54 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:20.425267 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca0ee4b-fd32-4bad-8dde-53fbac5d9b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:20.425267 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca0ee4b-fd32-4bad-8dde-53fbac5d9b54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:20.425267 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca0ee4b-fd32-4bad-8dde-53fbac5d9b54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:20.426651 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca0ee4b-fd32-4bad-8dde-53fbac5d9b54 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:20.438463 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 934/1884] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:04:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:04:20.442246 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a518f176-50df-429d-bfb2-b50472bb1817 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:20.457183 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a518f176-50df-429d-bfb2-b50472bb1817 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:20.464255 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6475b6d-8b72-4dc3-a62a-335e2c9ca84d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:20.465511 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6475b6d-8b72-4dc3-a62a-335e2c9ca84d admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:20.465857 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6475b6d-8b72-4dc3-a62a-335e2c9ca84d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:20.466139 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6475b6d-8b72-4dc3-a62a-335e2c9ca84d admin admin] PATH_INFO: `/v3/projects/009795ea810540a4b1458bcbaa5cb17a/users/056bcb6d5c664ed4aa3e50e6b233d5cd/roles/43106c664e1b40de984c0c62701149c8` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:20.484983 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6475b6d-8b72-4dc3-a62a-335e2c9ca84d admin admin] RBAC: Authorizing `identity:create_grant(project_id=009795ea810540a4b1458bcbaa5cb17a, user_id=056bcb6d5c664ed4aa3e50e6b233d5cd, role_id=43106c664e1b40de984c0c62701149c8)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:20.488443 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6475b6d-8b72-4dc3-a62a-335e2c9ca84d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:20.547791 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 951/1885] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:04:20 2022] PUT /identity/v3/projects/009795ea810540a4b1458bcbaa5cb17a/users/056bcb6d5c664ed4aa3e50e6b233d5cd/roles/43106c664e1b40de984c0c62701149c8 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:04:20.562474 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e92c667-4134-4fcf-bebb-3fe87fbd52c1 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:20.562797 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e92c667-4134-4fcf-bebb-3fe87fbd52c1 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:20.563024 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e92c667-4134-4fcf-bebb-3fe87fbd52c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:20.604137 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-7e92c667-4134-4fcf-bebb-3fe87fbd52c1 None None] MFA Rules not processed for user `056bcb6d5c664ed4aa3e50e6b233d5cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:04:20.606048 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-7e92c667-4134-4fcf-bebb-3fe87fbd52c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:20.696695 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:04:20.735930 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 935/1886] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:04:20 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:04:20.739511 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6475b6d-8b72-4dc3-a62a-335e2c9ca84d admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:20.815301 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6475b6d-8b72-4dc3-a62a-335e2c9ca84d admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:20.820322 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da69e55-bf14-4a1d-b635-fd32d56a03a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:20.821575 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da69e55-bf14-4a1d-b635-fd32d56a03a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:20.821861 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da69e55-bf14-4a1d-b635-fd32d56a03a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:20.822290 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da69e55-bf14-4a1d-b635-fd32d56a03a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:20.823180 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da69e55-bf14-4a1d-b635-fd32d56a03a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:20.826482 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da69e55-bf14-4a1d-b635-fd32d56a03a0 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:20.911203 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 952/1887] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:04:20 2022] POST /identity/v3/projects => generated 381 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:04:20.915077 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e92c667-4134-4fcf-bebb-3fe87fbd52c1 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:20.925799 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e92c667-4134-4fcf-bebb-3fe87fbd52c1 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:20.981497 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526bec82-31ff-4609-a447-2980b3885c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:20.982178 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526bec82-31ff-4609-a447-2980b3885c11 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:20.982465 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526bec82-31ff-4609-a447-2980b3885c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:20.982745 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526bec82-31ff-4609-a447-2980b3885c11 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:20.983448 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526bec82-31ff-4609-a447-2980b3885c11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:20.985490 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526bec82-31ff-4609-a447-2980b3885c11 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:21.073138 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 936/1888] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:04:20 2022] POST /identity/v3/users => generated 297 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:04:21.077012 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da69e55-bf14-4a1d-b635-fd32d56a03a0 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:21.088030 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da69e55-bf14-4a1d-b635-fd32d56a03a0 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:21.092747 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db0c6a2-0d0c-4060-be22-bf6e78e3b05a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:21.093530 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db0c6a2-0d0c-4060-be22-bf6e78e3b05a admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:21.093813 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db0c6a2-0d0c-4060-be22-bf6e78e3b05a admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:21.094275 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db0c6a2-0d0c-4060-be22-bf6e78e3b05a admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:21.095001 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db0c6a2-0d0c-4060-be22-bf6e78e3b05a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:21.096880 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db0c6a2-0d0c-4060-be22-bf6e78e3b05a admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:21.108900 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 953/1889] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:04:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:04:21.113891 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526bec82-31ff-4609-a447-2980b3885c11 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:21.126199 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526bec82-31ff-4609-a447-2980b3885c11 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:21.131700 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d21d822-295f-42bb-aa99-b37051c3e803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:21.132778 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d21d822-295f-42bb-aa99-b37051c3e803 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:21.133180 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d21d822-295f-42bb-aa99-b37051c3e803 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:21.138100 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d21d822-295f-42bb-aa99-b37051c3e803 admin admin] PATH_INFO: `/v3/projects/a495b4c4a5614697a1e4369a56380b27/users/89c2ab5796f54985ab90e5931ae2dfca/roles/b43d5a8247e244869c2c70f986f42e8a` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:21.159261 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d21d822-295f-42bb-aa99-b37051c3e803 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a495b4c4a5614697a1e4369a56380b27, user_id=89c2ab5796f54985ab90e5931ae2dfca, role_id=b43d5a8247e244869c2c70f986f42e8a)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:21.164150 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d21d822-295f-42bb-aa99-b37051c3e803 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:21.198563 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:04:21.255566 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 937/1890] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:04:21 2022] PUT /identity/v3/projects/a495b4c4a5614697a1e4369a56380b27/users/89c2ab5796f54985ab90e5931ae2dfca/roles/b43d5a8247e244869c2c70f986f42e8a => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:04:21.260283 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db0c6a2-0d0c-4060-be22-bf6e78e3b05a admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:21.277180 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db0c6a2-0d0c-4060-be22-bf6e78e3b05a admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:21.343301 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8412ec4-1c0a-4b68-a37a-4d390b788727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:21.343658 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8412ec4-1c0a-4b68-a37a-4d390b788727 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:21.344302 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8412ec4-1c0a-4b68-a37a-4d390b788727 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:21.345421 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8412ec4-1c0a-4b68-a37a-4d390b788727 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:21.346175 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8412ec4-1c0a-4b68-a37a-4d390b788727 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:21.348145 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8412ec4-1c0a-4b68-a37a-4d390b788727 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:21.359549 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 954/1891] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:04:21 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 10 13:04:21.365833 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d21d822-295f-42bb-aa99-b37051c3e803 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:21.381533 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d21d822-295f-42bb-aa99-b37051c3e803 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:21.388462 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcf5ad1-3f4d-4d75-ae6e-2ad80394ffdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:21.390027 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcf5ad1-3f4d-4d75-ae6e-2ad80394ffdf admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:21.390388 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcf5ad1-3f4d-4d75-ae6e-2ad80394ffdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:21.390747 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcf5ad1-3f4d-4d75-ae6e-2ad80394ffdf admin admin] PATH_INFO: `/v3/projects/a495b4c4a5614697a1e4369a56380b27/users/89c2ab5796f54985ab90e5931ae2dfca/roles/b43d5a8247e244869c2c70f986f42e8a` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:21.394047 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcf5ad1-3f4d-4d75-ae6e-2ad80394ffdf admin admin] RBAC: Authorizing `identity:create_grant(project_id=a495b4c4a5614697a1e4369a56380b27, user_id=89c2ab5796f54985ab90e5931ae2dfca, role_id=b43d5a8247e244869c2c70f986f42e8a)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:21.398328 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcf5ad1-3f4d-4d75-ae6e-2ad80394ffdf admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:21.412355 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 938/1892] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:04:21 2022] PUT /identity/v3/projects/a495b4c4a5614697a1e4369a56380b27/users/89c2ab5796f54985ab90e5931ae2dfca/roles/b43d5a8247e244869c2c70f986f42e8a => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:04:21.423567 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a13c4c-f985-436e-99b7-91d4d7948db2 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:21.424146 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a13c4c-f985-436e-99b7-91d4d7948db2 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:21.424585 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a13c4c-f985-436e-99b7-91d4d7948db2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:21.477811 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcf5ad1-3f4d-4d75-ae6e-2ad80394ffdf admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:21.480082 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0bcf5ad1-3f4d-4d75-ae6e-2ad80394ffdf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:21.485936 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-73a13c4c-f985-436e-99b7-91d4d7948db2 None None] MFA Rules not processed for user `89c2ab5796f54985ab90e5931ae2dfca`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:04:21.485936 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-73a13c4c-f985-436e-99b7-91d4d7948db2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:21.512411 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:04:21.560300 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcf5ad1-3f4d-4d75-ae6e-2ad80394ffdf admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:21.565022 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8789901-70cc-41b1-83c7-5b564a68dc42 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6cc95a563b941ac92ab283aa0d5a5d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:21.566109 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8789901-70cc-41b1-83c7-5b564a68dc42 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:21.568029 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8789901-70cc-41b1-83c7-5b564a68dc42 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:21.575038 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8789901-70cc-41b1-83c7-5b564a68dc42 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:21.575038 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a8789901-70cc-41b1-83c7-5b564a68dc42 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:21.612309 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 955/1893] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:04:21 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:04:21.620688 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a13c4c-f985-436e-99b7-91d4d7948db2 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:21.635823 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a13c4c-f985-436e-99b7-91d4d7948db2 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:21.642437 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29d460a-11c6-4790-9f6b-34dca7c5276f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6cc95a563b941ac92ab283aa0d5a5d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:21.643812 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29d460a-11c6-4790-9f6b-34dca7c5276f service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:21.643812 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29d460a-11c6-4790-9f6b-34dca7c5276f service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:21.644118 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29d460a-11c6-4790-9f6b-34dca7c5276f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:21.657997 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29d460a-11c6-4790-9f6b-34dca7c5276f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:21.660095 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29d460a-11c6-4790-9f6b-34dca7c5276f service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:21.660241 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8789901-70cc-41b1-83c7-5b564a68dc42 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:21.661840 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8789901-70cc-41b1-83c7-5b564a68dc42 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:21.673880 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 939/1894] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:04:21 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:04:21.674476 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 956/1895] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:04:21 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:04:21.776832 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:04:21.835623 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8789901-70cc-41b1-83c7-5b564a68dc42 service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:21.840491 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:04:21.846631 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8789901-70cc-41b1-83c7-5b564a68dc42 service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:21.851429 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847c0431-5b0c-47df-a6da-c620fc4d3e85 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6cc95a563b941ac92ab283aa0d5a5d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:21.852923 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847c0431-5b0c-47df-a6da-c620fc4d3e85 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:21.853399 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847c0431-5b0c-47df-a6da-c620fc4d3e85 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:21.854087 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847c0431-5b0c-47df-a6da-c620fc4d3e85 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:21.952735 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847c0431-5b0c-47df-a6da-c620fc4d3e85 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:21.954337 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847c0431-5b0c-47df-a6da-c620fc4d3e85 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:21.977062 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 940/1896] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:04:21 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:04:22.118868 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:04:22.212415 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:04:22.588086 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:04:22.605815 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29d460a-11c6-4790-9f6b-34dca7c5276f service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:22.694360 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29d460a-11c6-4790-9f6b-34dca7c5276f service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:22.700243 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a657cbb7-cdbb-4e8b-a0f4-52516e360da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:22.701206 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a657cbb7-cdbb-4e8b-a0f4-52516e360da9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:22.701613 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a657cbb7-cdbb-4e8b-a0f4-52516e360da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:22.701950 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a657cbb7-cdbb-4e8b-a0f4-52516e360da9 admin admin] PATH_INFO: `/v3/users/056bcb6d5c664ed4aa3e50e6b233d5cd` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:22.703614 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a657cbb7-cdbb-4e8b-a0f4-52516e360da9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=056bcb6d5c664ed4aa3e50e6b233d5cd)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:22.705345 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a657cbb7-cdbb-4e8b-a0f4-52516e360da9 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:22.861244 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:04:22.930021 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:04:22.943056 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a657cbb7-cdbb-4e8b-a0f4-52516e360da9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '056bcb6d5c664ed4aa3e50e6b233d5cd'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:22.950337 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a657cbb7-cdbb-4e8b-a0f4-52516e360da9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '056bcb6d5c664ed4aa3e50e6b233d5cd'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:23.038348 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a657cbb7-cdbb-4e8b-a0f4-52516e360da9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '056bcb6d5c664ed4aa3e50e6b233d5cd'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:23.039044 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a657cbb7-cdbb-4e8b-a0f4-52516e360da9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '056bcb6d5c664ed4aa3e50e6b233d5cd'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:23.053740 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a657cbb7-cdbb-4e8b-a0f4-52516e360da9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '056bcb6d5c664ed4aa3e50e6b233d5cd'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:23.060997 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 957/1897] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:04:22 2022] DELETE /identity/v3/users/056bcb6d5c664ed4aa3e50e6b233d5cd => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:04:23.065476 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67270416-ef64-4843-9a37-000204968150 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:23.065767 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67270416-ef64-4843-9a37-000204968150 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:23.066243 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67270416-ef64-4843-9a37-000204968150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:23.084600 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-67270416-ef64-4843-9a37-000204968150 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:04:23.086327 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-67270416-ef64-4843-9a37-000204968150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:23.161156 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 941/1898] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:04:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:04:23.171426 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a657cbb7-cdbb-4e8b-a0f4-52516e360da9 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:23.172902 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a657cbb7-cdbb-4e8b-a0f4-52516e360da9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:23.229483 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a657cbb7-cdbb-4e8b-a0f4-52516e360da9 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:23.233290 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86903f7a-1e55-41c5-91d2-78cbe7c17f38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:23.233928 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86903f7a-1e55-41c5-91d2-78cbe7c17f38 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:23.234207 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86903f7a-1e55-41c5-91d2-78cbe7c17f38 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:23.234487 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86903f7a-1e55-41c5-91d2-78cbe7c17f38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:23.244061 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86903f7a-1e55-41c5-91d2-78cbe7c17f38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:23.245467 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86903f7a-1e55-41c5-91d2-78cbe7c17f38 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:23.255024 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 958/1899] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:04:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:04:23.748163 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67270416-ef64-4843-9a37-000204968150 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:23.749569 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-67270416-ef64-4843-9a37-000204968150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:23.764048 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67270416-ef64-4843-9a37-000204968150 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:23.768832 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ce8414-9271-44f8-8c7b-e2ab368e61b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:23.769703 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ce8414-9271-44f8-8c7b-e2ab368e61b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:23.769985 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ce8414-9271-44f8-8c7b-e2ab368e61b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:23.770261 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ce8414-9271-44f8-8c7b-e2ab368e61b4 admin admin] PATH_INFO: `/v3/projects/009795ea810540a4b1458bcbaa5cb17a` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:23.771474 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ce8414-9271-44f8-8c7b-e2ab368e61b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=009795ea810540a4b1458bcbaa5cb17a)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:23.773672 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ce8414-9271-44f8-8c7b-e2ab368e61b4 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:23.879295 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-64ce8414-9271-44f8-8c7b-e2ab368e61b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '009795ea810540a4b1458bcbaa5cb17a', 'request_id': 'req-64ce8414-9271-44f8-8c7b-e2ab368e61b4'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:23.885863 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-64ce8414-9271-44f8-8c7b-e2ab368e61b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '009795ea810540a4b1458bcbaa5cb17a', 'request_id': 'req-64ce8414-9271-44f8-8c7b-e2ab368e61b4'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:23.893864 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-64ce8414-9271-44f8-8c7b-e2ab368e61b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '009795ea810540a4b1458bcbaa5cb17a', 'request_id': 'req-64ce8414-9271-44f8-8c7b-e2ab368e61b4'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:23.969965 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-64ce8414-9271-44f8-8c7b-e2ab368e61b4 admin admin] The token cache is being invalidate because project 009795ea810540a4b1458bcbaa5cb17a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:04:23.970261 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-64ce8414-9271-44f8-8c7b-e2ab368e61b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:23.970887 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-64ce8414-9271-44f8-8c7b-e2ab368e61b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:23.972475 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 942/1900] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:04:23 2022] DELETE /identity/v3/projects/009795ea810540a4b1458bcbaa5cb17a => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:04:23.976761 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86903f7a-1e55-41c5-91d2-78cbe7c17f38 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:23.986807 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86903f7a-1e55-41c5-91d2-78cbe7c17f38 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:24.048102 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f40e52f-f4cd-4524-87d9-bb453dfd7eea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:24.048985 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f40e52f-f4cd-4524-87d9-bb453dfd7eea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:24.049270 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f40e52f-f4cd-4524-87d9-bb453dfd7eea admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:24.049564 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f40e52f-f4cd-4524-87d9-bb453dfd7eea admin admin] PATH_INFO: `/v3/users/89c2ab5796f54985ab90e5931ae2dfca` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:24.050933 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f40e52f-f4cd-4524-87d9-bb453dfd7eea admin admin] RBAC: Authorizing `identity:delete_user(user_id=89c2ab5796f54985ab90e5931ae2dfca)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:24.052936 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f40e52f-f4cd-4524-87d9-bb453dfd7eea admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:24.060051 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:04:24.233993 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4f40e52f-f4cd-4524-87d9-bb453dfd7eea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89c2ab5796f54985ab90e5931ae2dfca'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:24.241383 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4f40e52f-f4cd-4524-87d9-bb453dfd7eea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89c2ab5796f54985ab90e5931ae2dfca'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:24.317855 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4f40e52f-f4cd-4524-87d9-bb453dfd7eea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89c2ab5796f54985ab90e5931ae2dfca'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:24.318478 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4f40e52f-f4cd-4524-87d9-bb453dfd7eea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89c2ab5796f54985ab90e5931ae2dfca'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:24.332487 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4f40e52f-f4cd-4524-87d9-bb453dfd7eea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89c2ab5796f54985ab90e5931ae2dfca'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:24.338782 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 959/1901] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:04:23 2022] DELETE /identity/v3/users/89c2ab5796f54985ab90e5931ae2dfca => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 10 13:04:24.990230 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ce8414-9271-44f8-8c7b-e2ab368e61b4 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:24.991847 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-64ce8414-9271-44f8-8c7b-e2ab368e61b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:25.007087 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ce8414-9271-44f8-8c7b-e2ab368e61b4 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:25.017858 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:04:25.070768 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b447e613-8ca8-4a55-9f0b-49fa453694b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:25.071835 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b447e613-8ca8-4a55-9f0b-49fa453694b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:25.072147 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b447e613-8ca8-4a55-9f0b-49fa453694b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:25.072423 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b447e613-8ca8-4a55-9f0b-49fa453694b2 admin admin] PATH_INFO: `/v3/projects/a495b4c4a5614697a1e4369a56380b27` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:25.073623 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b447e613-8ca8-4a55-9f0b-49fa453694b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a495b4c4a5614697a1e4369a56380b27)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:25.075645 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b447e613-8ca8-4a55-9f0b-49fa453694b2 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:25.228022 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b447e613-8ca8-4a55-9f0b-49fa453694b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a495b4c4a5614697a1e4369a56380b27', 'request_id': 'req-b447e613-8ca8-4a55-9f0b-49fa453694b2'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:25.234695 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b447e613-8ca8-4a55-9f0b-49fa453694b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a495b4c4a5614697a1e4369a56380b27', 'request_id': 'req-b447e613-8ca8-4a55-9f0b-49fa453694b2'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:25.241865 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b447e613-8ca8-4a55-9f0b-49fa453694b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a495b4c4a5614697a1e4369a56380b27', 'request_id': 'req-b447e613-8ca8-4a55-9f0b-49fa453694b2'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:25.357546 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b447e613-8ca8-4a55-9f0b-49fa453694b2 admin admin] The token cache is being invalidate because project a495b4c4a5614697a1e4369a56380b27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:04:25.358141 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b447e613-8ca8-4a55-9f0b-49fa453694b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:25.358936 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b447e613-8ca8-4a55-9f0b-49fa453694b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:25.365292 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 943/1902] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:04:24 2022] DELETE /identity/v3/projects/a495b4c4a5614697a1e4369a56380b27 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:04:25.367304 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee58cf2-3d27-477d-b908-85580d65bbc8 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:25.367793 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee58cf2-3d27-477d-b908-85580d65bbc8 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:25.369937 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee58cf2-3d27-477d-b908-85580d65bbc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:25.394836 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-9ee58cf2-3d27-477d-b908-85580d65bbc8 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:04:25.396137 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9ee58cf2-3d27-477d-b908-85580d65bbc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:25.477425 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 960/1903] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:04:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 10 13:04:25.485473 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628ca1aa-d62f-4be0-a3ce-255c07fad72e None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:25.486098 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628ca1aa-d62f-4be0-a3ce-255c07fad72e None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:25.486580 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628ca1aa-d62f-4be0-a3ce-255c07fad72e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:25.506710 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-628ca1aa-d62f-4be0-a3ce-255c07fad72e None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:04:25.507977 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-628ca1aa-d62f-4be0-a3ce-255c07fad72e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:25.518132 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 944/1904] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:04:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:04:25.522498 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee58cf2-3d27-477d-b908-85580d65bbc8 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:25.533063 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee58cf2-3d27-477d-b908-85580d65bbc8 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:25.538207 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcc6bfd-c5b4-4d5e-91f8-6ee783d1e556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:25.538887 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcc6bfd-c5b4-4d5e-91f8-6ee783d1e556 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:25.539180 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcc6bfd-c5b4-4d5e-91f8-6ee783d1e556 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:25.539492 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcc6bfd-c5b4-4d5e-91f8-6ee783d1e556 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:25.540274 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcc6bfd-c5b4-4d5e-91f8-6ee783d1e556 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 13:04:25.540587 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcc6bfd-c5b4-4d5e-91f8-6ee783d1e556 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:25.542185 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcc6bfd-c5b4-4d5e-91f8-6ee783d1e556 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:25.554313 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 961/1905] 10.222.0.27 () {58 vars in 1152 bytes} [Fri Jun 10 13:04:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 13:04:25.558784 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628ca1aa-d62f-4be0-a3ce-255c07fad72e None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:25.570083 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628ca1aa-d62f-4be0-a3ce-255c07fad72e None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:25.583495 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e43c266-d888-4e8d-8eb1-0ec865b24e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:25.588636 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e43c266-d888-4e8d-8eb1-0ec865b24e18 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:25.588636 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e43c266-d888-4e8d-8eb1-0ec865b24e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:25.588636 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e43c266-d888-4e8d-8eb1-0ec865b24e18 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:25.588636 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e43c266-d888-4e8d-8eb1-0ec865b24e18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:25.589479 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e43c266-d888-4e8d-8eb1-0ec865b24e18 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:25.696178 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 945/1906] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:04:25 2022] POST /identity/v3/projects => generated 377 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 10 13:04:25.700587 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcc6bfd-c5b4-4d5e-91f8-6ee783d1e556 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:25.713076 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcc6bfd-c5b4-4d5e-91f8-6ee783d1e556 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:25.786377 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24456a44-bc61-4a5e-a750-5861192c35a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:25.787263 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24456a44-bc61-4a5e-a750-5861192c35a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:25.787547 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24456a44-bc61-4a5e-a750-5861192c35a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:25.787932 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24456a44-bc61-4a5e-a750-5861192c35a0 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:25.789001 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24456a44-bc61-4a5e-a750-5861192c35a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:25.792577 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24456a44-bc61-4a5e-a750-5861192c35a0 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:25.906555 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 962/1907] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:04:25 2022] POST /identity/v3/users => generated 289 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:04:25.910802 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e43c266-d888-4e8d-8eb1-0ec865b24e18 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:25.925775 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e43c266-d888-4e8d-8eb1-0ec865b24e18 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:25.932749 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81254e03-c9e7-4bf3-95be-9d2863169915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:25.933841 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81254e03-c9e7-4bf3-95be-9d2863169915 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:25.934285 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81254e03-c9e7-4bf3-95be-9d2863169915 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:25.934591 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81254e03-c9e7-4bf3-95be-9d2863169915 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:25.935309 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81254e03-c9e7-4bf3-95be-9d2863169915 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:25.937314 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81254e03-c9e7-4bf3-95be-9d2863169915 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:25.953711 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 946/1908] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:04:25 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 10 13:04:25.955322 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24456a44-bc61-4a5e-a750-5861192c35a0 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:25.961123 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:04:25.968130 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24456a44-bc61-4a5e-a750-5861192c35a0 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:25.973742 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a5b8ed-ccf3-4537-b032-e218d90adffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:25.974842 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a5b8ed-ccf3-4537-b032-e218d90adffd admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:25.975144 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a5b8ed-ccf3-4537-b032-e218d90adffd admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:25.975686 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a5b8ed-ccf3-4537-b032-e218d90adffd admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:25.976467 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a5b8ed-ccf3-4537-b032-e218d90adffd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:25.978260 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a5b8ed-ccf3-4537-b032-e218d90adffd admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:25.996142 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 963/1909] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:04:25 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:04:25.999189 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81254e03-c9e7-4bf3-95be-9d2863169915 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:26.011097 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81254e03-c9e7-4bf3-95be-9d2863169915 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:26.018007 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360f99f6-4a2b-41f9-9c78-214142f0ac4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:26.022720 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360f99f6-4a2b-41f9-9c78-214142f0ac4d admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:26.022720 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360f99f6-4a2b-41f9-9c78-214142f0ac4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:26.022720 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360f99f6-4a2b-41f9-9c78-214142f0ac4d admin admin] PATH_INFO: `/v3/projects/72116263e3af4375976c608558f8967b/users/742567443239416280ed71f2297b850a/roles/43106c664e1b40de984c0c62701149c8` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:26.052020 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360f99f6-4a2b-41f9-9c78-214142f0ac4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=72116263e3af4375976c608558f8967b, user_id=742567443239416280ed71f2297b850a, role_id=43106c664e1b40de984c0c62701149c8)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:26.067467 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360f99f6-4a2b-41f9-9c78-214142f0ac4d admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:26.160784 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 947/1910] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:04:25 2022] PUT /identity/v3/projects/72116263e3af4375976c608558f8967b/users/742567443239416280ed71f2297b850a/roles/43106c664e1b40de984c0c62701149c8 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:04:26.172094 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760ab153-2ab6-4227-94cb-04df701c2f95 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:26.172386 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760ab153-2ab6-4227-94cb-04df701c2f95 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:26.177067 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760ab153-2ab6-4227-94cb-04df701c2f95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:26.204763 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-760ab153-2ab6-4227-94cb-04df701c2f95 None None] MFA Rules not processed for user `742567443239416280ed71f2297b850a`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:04:26.205996 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-760ab153-2ab6-4227-94cb-04df701c2f95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:26.300400 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 964/1911] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:04:26 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:04:26.316777 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360f99f6-4a2b-41f9-9c78-214142f0ac4d admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:26.319297 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-360f99f6-4a2b-41f9-9c78-214142f0ac4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:26.379361 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:04:26.398029 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360f99f6-4a2b-41f9-9c78-214142f0ac4d admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:26.402163 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dcdc0e-855c-4b9b-8185-995baa3c875c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d3eb01a98274050b930616b98998c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:26.402811 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dcdc0e-855c-4b9b-8185-995baa3c875c service glance] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:26.403355 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dcdc0e-855c-4b9b-8185-995baa3c875c service glance] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:26.403821 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dcdc0e-855c-4b9b-8185-995baa3c875c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:26.413304 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dcdc0e-855c-4b9b-8185-995baa3c875c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:26.415498 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dcdc0e-855c-4b9b-8185-995baa3c875c service glance] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:26.426990 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 948/1912] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:04:26 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:04:28.787143 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760ab153-2ab6-4227-94cb-04df701c2f95 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:28.791512 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-760ab153-2ab6-4227-94cb-04df701c2f95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:28.887234 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760ab153-2ab6-4227-94cb-04df701c2f95 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:28.891855 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4f3c47-543e-4564-8cd2-b02c522ced3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:28.892755 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4f3c47-543e-4564-8cd2-b02c522ced3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:28.893033 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4f3c47-543e-4564-8cd2-b02c522ced3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:28.893428 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4f3c47-543e-4564-8cd2-b02c522ced3f admin admin] PATH_INFO: `/v3/users/6f46e18c70724dada243fde343e20eb7` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:28.894540 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4f3c47-543e-4564-8cd2-b02c522ced3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f46e18c70724dada243fde343e20eb7)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:28.896519 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4f3c47-543e-4564-8cd2-b02c522ced3f admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:29.108024 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2e4f3c47-543e-4564-8cd2-b02c522ced3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f46e18c70724dada243fde343e20eb7'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:29.114878 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2e4f3c47-543e-4564-8cd2-b02c522ced3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f46e18c70724dada243fde343e20eb7'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:29.224006 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dcdc0e-855c-4b9b-8185-995baa3c875c service glance] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:29.225425 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b5dcdc0e-855c-4b9b-8185-995baa3c875c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:29.252867 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2e4f3c47-543e-4564-8cd2-b02c522ced3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f46e18c70724dada243fde343e20eb7'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:29.253406 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2e4f3c47-543e-4564-8cd2-b02c522ced3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f46e18c70724dada243fde343e20eb7'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:29.274492 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2e4f3c47-543e-4564-8cd2-b02c522ced3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f46e18c70724dada243fde343e20eb7'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:29.288634 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:04:29.294656 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 965/1913] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:04:28 2022] DELETE /identity/v3/users/6f46e18c70724dada243fde343e20eb7 => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:04:29.299622 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e974e020-eb10-4f69-ad3f-646fb54f8a90 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:29.300078 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e974e020-eb10-4f69-ad3f-646fb54f8a90 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:29.300435 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e974e020-eb10-4f69-ad3f-646fb54f8a90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:29.312763 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dcdc0e-855c-4b9b-8185-995baa3c875c service glance] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:29.320426 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-e974e020-eb10-4f69-ad3f-646fb54f8a90 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:04:29.321676 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e974e020-eb10-4f69-ad3f-646fb54f8a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:29.368693 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df51019-da46-4e99-99b0-0d08109d30ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6cc95a563b941ac92ab283aa0d5a5d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:29.369499 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df51019-da46-4e99-99b0-0d08109d30ad service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:29.369835 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df51019-da46-4e99-99b0-0d08109d30ad service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:29.370119 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df51019-da46-4e99-99b0-0d08109d30ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:29.372272 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-1df51019-da46-4e99-99b0-0d08109d30ad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:29.393819 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 966/1914] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:04:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:04:29.404136 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e974e020-eb10-4f69-ad3f-646fb54f8a90 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:29.406166 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e974e020-eb10-4f69-ad3f-646fb54f8a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:29.446392 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df51019-da46-4e99-99b0-0d08109d30ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:29.447893 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df51019-da46-4e99-99b0-0d08109d30ad service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:29.458175 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 949/1915] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:04:29 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 10 13:04:29.469941 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e974e020-eb10-4f69-ad3f-646fb54f8a90 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:29.473863 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d40422d-ea58-446a-9560-feef7ef49718 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:29.474566 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d40422d-ea58-446a-9560-feef7ef49718 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:29.474854 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d40422d-ea58-446a-9560-feef7ef49718 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:29.475133 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d40422d-ea58-446a-9560-feef7ef49718 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:29.486759 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d40422d-ea58-446a-9560-feef7ef49718 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:29.489005 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d40422d-ea58-446a-9560-feef7ef49718 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:29.500100 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 967/1916] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:04:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 10 13:04:30.017539 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df51019-da46-4e99-99b0-0d08109d30ad service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:30.018790 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-1df51019-da46-4e99-99b0-0d08109d30ad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:30.048066 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df51019-da46-4e99-99b0-0d08109d30ad service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:30.065318 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f224918-aa76-479d-b062-f592addd7219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:30.066462 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f224918-aa76-479d-b062-f592addd7219 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:30.066998 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f224918-aa76-479d-b062-f592addd7219 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:30.067395 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f224918-aa76-479d-b062-f592addd7219 admin admin] PATH_INFO: `/v3/projects/5629f339b5554aeaab8bea8a33ac21a4` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:30.069198 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f224918-aa76-479d-b062-f592addd7219 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5629f339b5554aeaab8bea8a33ac21a4)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:30.072779 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f224918-aa76-479d-b062-f592addd7219 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:30.196916 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:04:30.200010 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-1f224918-aa76-479d-b062-f592addd7219 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5629f339b5554aeaab8bea8a33ac21a4', 'request_id': 'req-1f224918-aa76-479d-b062-f592addd7219'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:30.206045 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-1f224918-aa76-479d-b062-f592addd7219 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5629f339b5554aeaab8bea8a33ac21a4', 'request_id': 'req-1f224918-aa76-479d-b062-f592addd7219'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:30.213287 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-1f224918-aa76-479d-b062-f592addd7219 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5629f339b5554aeaab8bea8a33ac21a4', 'request_id': 'req-1f224918-aa76-479d-b062-f592addd7219'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:30.376910 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-1f224918-aa76-479d-b062-f592addd7219 admin admin] The token cache is being invalidate because project 5629f339b5554aeaab8bea8a33ac21a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:04:30.377527 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-1f224918-aa76-479d-b062-f592addd7219 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:30.378043 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-1f224918-aa76-479d-b062-f592addd7219 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:30.379822 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 950/1917] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:04:30 2022] DELETE /identity/v3/projects/5629f339b5554aeaab8bea8a33ac21a4 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:04:30.384631 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d40422d-ea58-446a-9560-feef7ef49718 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:30.394981 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d40422d-ea58-446a-9560-feef7ef49718 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:30.475763 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e0f843-2713-4062-9159-ddcc646f66c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:30.476818 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e0f843-2713-4062-9159-ddcc646f66c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:30.477715 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e0f843-2713-4062-9159-ddcc646f66c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:30.478002 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e0f843-2713-4062-9159-ddcc646f66c4 admin admin] PATH_INFO: `/v3/users/8723b893a99a45879b2e28a54d5c7a28` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:30.479749 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e0f843-2713-4062-9159-ddcc646f66c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8723b893a99a45879b2e28a54d5c7a28)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:30.481825 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e0f843-2713-4062-9159-ddcc646f66c4 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:30.739489 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:04:30.741264 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b1e0f843-2713-4062-9159-ddcc646f66c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8723b893a99a45879b2e28a54d5c7a28'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:30.749254 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b1e0f843-2713-4062-9159-ddcc646f66c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8723b893a99a45879b2e28a54d5c7a28'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:30.832212 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b1e0f843-2713-4062-9159-ddcc646f66c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8723b893a99a45879b2e28a54d5c7a28'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:30.834025 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b1e0f843-2713-4062-9159-ddcc646f66c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8723b893a99a45879b2e28a54d5c7a28'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:30.855184 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b1e0f843-2713-4062-9159-ddcc646f66c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8723b893a99a45879b2e28a54d5c7a28'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:30.861381 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 968/1918] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:04:30 2022] DELETE /identity/v3/users/8723b893a99a45879b2e28a54d5c7a28 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 10 13:04:31.427371 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f224918-aa76-479d-b062-f592addd7219 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:31.427371 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-1f224918-aa76-479d-b062-f592addd7219 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:31.461508 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f224918-aa76-479d-b062-f592addd7219 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:31.572440 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99f7355-0229-44ea-9bdd-706c6a8a4bcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:31.572773 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99f7355-0229-44ea-9bdd-706c6a8a4bcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:31.573058 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99f7355-0229-44ea-9bdd-706c6a8a4bcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:31.573338 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99f7355-0229-44ea-9bdd-706c6a8a4bcb admin admin] PATH_INFO: `/v3/projects/884ca11abc3540a79deffdea80c40232` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:31.574546 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99f7355-0229-44ea-9bdd-706c6a8a4bcb admin admin] RBAC: Authorizing `identity:delete_project(project_id=884ca11abc3540a79deffdea80c40232)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:31.576509 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99f7355-0229-44ea-9bdd-706c6a8a4bcb admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:31.699893 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-e99f7355-0229-44ea-9bdd-706c6a8a4bcb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '884ca11abc3540a79deffdea80c40232', 'request_id': 'req-e99f7355-0229-44ea-9bdd-706c6a8a4bcb'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:31.708381 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-e99f7355-0229-44ea-9bdd-706c6a8a4bcb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '884ca11abc3540a79deffdea80c40232', 'request_id': 'req-e99f7355-0229-44ea-9bdd-706c6a8a4bcb'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:31.720047 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-e99f7355-0229-44ea-9bdd-706c6a8a4bcb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '884ca11abc3540a79deffdea80c40232', 'request_id': 'req-e99f7355-0229-44ea-9bdd-706c6a8a4bcb'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:31.798113 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-e99f7355-0229-44ea-9bdd-706c6a8a4bcb admin admin] The token cache is being invalidate because project 884ca11abc3540a79deffdea80c40232 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:04:31.798971 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-e99f7355-0229-44ea-9bdd-706c6a8a4bcb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:31.800001 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-e99f7355-0229-44ea-9bdd-706c6a8a4bcb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:31.802130 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 951/1919] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:04:31 2022] DELETE /identity/v3/projects/884ca11abc3540a79deffdea80c40232 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:04:31.807330 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421dee02-3cdf-417b-8302-3e09880f4add None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:31.807812 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421dee02-3cdf-417b-8302-3e09880f4add None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:31.807986 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421dee02-3cdf-417b-8302-3e09880f4add None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:31.826653 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-421dee02-3cdf-417b-8302-3e09880f4add None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:04:31.828232 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-421dee02-3cdf-417b-8302-3e09880f4add None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:31.942978 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 969/1920] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:04:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:04:31.951122 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260677dc-868d-4a00-b90f-c30aa52da0b7 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:31.951765 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260677dc-868d-4a00-b90f-c30aa52da0b7 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:31.952541 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260677dc-868d-4a00-b90f-c30aa52da0b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:31.983574 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-260677dc-868d-4a00-b90f-c30aa52da0b7 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:04:31.986107 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-260677dc-868d-4a00-b90f-c30aa52da0b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:32.003724 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 952/1921] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:04:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:04:32.009504 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421dee02-3cdf-417b-8302-3e09880f4add None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:32.022334 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421dee02-3cdf-417b-8302-3e09880f4add None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:32.028281 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daef0358-aa51-4df1-a980-df9467822c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:32.029294 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef0358-aa51-4df1-a980-df9467822c97 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:32.029769 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef0358-aa51-4df1-a980-df9467822c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:32.030229 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef0358-aa51-4df1-a980-df9467822c97 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:32.031155 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daef0358-aa51-4df1-a980-df9467822c97 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 13:04:32.031649 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daef0358-aa51-4df1-a980-df9467822c97 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:32.033809 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daef0358-aa51-4df1-a980-df9467822c97 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:32.052758 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260677dc-868d-4a00-b90f-c30aa52da0b7 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:32.056079 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 970/1922] 10.222.0.27 () {58 vars in 1152 bytes} [Fri Jun 10 13:04:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 13:04:32.064393 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260677dc-868d-4a00-b90f-c30aa52da0b7 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:32.070888 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f283fa2c-e18b-4f22-a5de-b161899e23c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:32.071574 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f283fa2c-e18b-4f22-a5de-b161899e23c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:32.072030 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f283fa2c-e18b-4f22-a5de-b161899e23c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:32.072317 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f283fa2c-e18b-4f22-a5de-b161899e23c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:32.073019 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f283fa2c-e18b-4f22-a5de-b161899e23c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:32.075063 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f283fa2c-e18b-4f22-a5de-b161899e23c7 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:32.199976 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 953/1923] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:04:32 2022] POST /identity/v3/projects => generated 363 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:04:32.210232 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daef0358-aa51-4df1-a980-df9467822c97 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:32.214370 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daef0358-aa51-4df1-a980-df9467822c97 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:32.282566 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15ff22b-f7bd-46fa-8b2d-5bd445616418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:32.282566 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15ff22b-f7bd-46fa-8b2d-5bd445616418 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:32.282566 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15ff22b-f7bd-46fa-8b2d-5bd445616418 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:32.282566 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15ff22b-f7bd-46fa-8b2d-5bd445616418 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:32.282932 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15ff22b-f7bd-46fa-8b2d-5bd445616418 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:32.283532 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15ff22b-f7bd-46fa-8b2d-5bd445616418 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:32.410636 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 971/1924] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:04:32 2022] POST /identity/v3/users => generated 282 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 10 13:04:32.411983 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f283fa2c-e18b-4f22-a5de-b161899e23c7 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:32.422838 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f283fa2c-e18b-4f22-a5de-b161899e23c7 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:32.428491 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96428580-58a4-439a-a4c0-5a89e0ae42cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:32.429238 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96428580-58a4-439a-a4c0-5a89e0ae42cd admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:32.429523 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96428580-58a4-439a-a4c0-5a89e0ae42cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:32.430374 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96428580-58a4-439a-a4c0-5a89e0ae42cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:32.431136 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96428580-58a4-439a-a4c0-5a89e0ae42cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:32.432936 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96428580-58a4-439a-a4c0-5a89e0ae42cd admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:32.445618 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 954/1925] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:04:32 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:04:32.449670 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15ff22b-f7bd-46fa-8b2d-5bd445616418 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:32.461331 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15ff22b-f7bd-46fa-8b2d-5bd445616418 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:32.466030 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b602e06c-4150-499d-975f-c57fda492b7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:32.466691 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b602e06c-4150-499d-975f-c57fda492b7b admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:32.466972 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b602e06c-4150-499d-975f-c57fda492b7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:32.467299 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b602e06c-4150-499d-975f-c57fda492b7b admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:32.468534 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b602e06c-4150-499d-975f-c57fda492b7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:32.470141 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b602e06c-4150-499d-975f-c57fda492b7b admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:32.480999 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 972/1926] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:04:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 10 13:04:32.485382 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96428580-58a4-439a-a4c0-5a89e0ae42cd admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:32.495911 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96428580-58a4-439a-a4c0-5a89e0ae42cd admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:32.500617 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b35e40-aa80-4c52-a8c2-6e118cbd616c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:32.501495 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b35e40-aa80-4c52-a8c2-6e118cbd616c admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:32.501779 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b35e40-aa80-4c52-a8c2-6e118cbd616c admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:32.502054 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b35e40-aa80-4c52-a8c2-6e118cbd616c admin admin] PATH_INFO: `/v3/projects/9c270c03baff4b448da6253c05a1e295/users/5dc2ece2db0d460d99f10bf695982335/roles/43106c664e1b40de984c0c62701149c8` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:32.520061 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b35e40-aa80-4c52-a8c2-6e118cbd616c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c270c03baff4b448da6253c05a1e295, user_id=5dc2ece2db0d460d99f10bf695982335, role_id=43106c664e1b40de984c0c62701149c8)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:32.522787 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b35e40-aa80-4c52-a8c2-6e118cbd616c admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:32.604071 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:04:32.612022 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 955/1927] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:04:32 2022] PUT /identity/v3/projects/9c270c03baff4b448da6253c05a1e295/users/5dc2ece2db0d460d99f10bf695982335/roles/43106c664e1b40de984c0c62701149c8 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:04:32.625253 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f79376-6a35-4230-bf52-08699b58ada1 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:32.625862 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f79376-6a35-4230-bf52-08699b58ada1 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:32.626305 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f79376-6a35-4230-bf52-08699b58ada1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:32.657455 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-71f79376-6a35-4230-bf52-08699b58ada1 None None] MFA Rules not processed for user `5dc2ece2db0d460d99f10bf695982335`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:04:32.658933 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-71f79376-6a35-4230-bf52-08699b58ada1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:32.679548 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:04:32.748690 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 973/1928] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:04:32 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 10 13:04:32.753314 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b35e40-aa80-4c52-a8c2-6e118cbd616c admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:32.823545 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b35e40-aa80-4c52-a8c2-6e118cbd616c admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:32.828278 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4daf8a8-1b9c-4a03-80d0-03563d0d68fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:32.828942 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4daf8a8-1b9c-4a03-80d0-03563d0d68fe admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:32.829228 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4daf8a8-1b9c-4a03-80d0-03563d0d68fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:32.829504 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4daf8a8-1b9c-4a03-80d0-03563d0d68fe admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:32.830195 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4daf8a8-1b9c-4a03-80d0-03563d0d68fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:32.832198 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4daf8a8-1b9c-4a03-80d0-03563d0d68fe admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:32.926109 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 956/1929] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:04:32 2022] POST /identity/v3/projects => generated 365 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 10 13:04:32.930231 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f79376-6a35-4230-bf52-08699b58ada1 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:32.943841 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f79376-6a35-4230-bf52-08699b58ada1 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:33.006189 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dc954b-8bd8-4995-811d-384340b67cf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:33.006779 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dc954b-8bd8-4995-811d-384340b67cf1 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:33.007089 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dc954b-8bd8-4995-811d-384340b67cf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:33.007343 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dc954b-8bd8-4995-811d-384340b67cf1 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:33.009248 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dc954b-8bd8-4995-811d-384340b67cf1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:33.011181 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dc954b-8bd8-4995-811d-384340b67cf1 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:33.119249 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 974/1930] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:04:32 2022] POST /identity/v3/users => generated 289 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:04:33.123382 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4daf8a8-1b9c-4a03-80d0-03563d0d68fe admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:33.137354 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4daf8a8-1b9c-4a03-80d0-03563d0d68fe admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:33.142297 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccad23e2-26a9-44cd-9605-de7ce2b05911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:33.142971 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccad23e2-26a9-44cd-9605-de7ce2b05911 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:33.143262 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccad23e2-26a9-44cd-9605-de7ce2b05911 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:33.143576 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccad23e2-26a9-44cd-9605-de7ce2b05911 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:33.144359 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccad23e2-26a9-44cd-9605-de7ce2b05911 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:33.145958 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccad23e2-26a9-44cd-9605-de7ce2b05911 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:33.158449 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 957/1931] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:04:33 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 10 13:04:33.162703 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dc954b-8bd8-4995-811d-384340b67cf1 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:33.173210 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dc954b-8bd8-4995-811d-384340b67cf1 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:33.177925 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc69a1d-112e-4eff-9a92-1287a41912fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:33.178849 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc69a1d-112e-4eff-9a92-1287a41912fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:33.179134 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc69a1d-112e-4eff-9a92-1287a41912fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:33.179436 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc69a1d-112e-4eff-9a92-1287a41912fd admin admin] PATH_INFO: `/v3/projects/92b1ea6f1bef4cc7a8d0abe3e619142e/users/c50464c435524fb38747ef4e650e7aac/roles/b43d5a8247e244869c2c70f986f42e8a` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:33.201580 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc69a1d-112e-4eff-9a92-1287a41912fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=92b1ea6f1bef4cc7a8d0abe3e619142e, user_id=c50464c435524fb38747ef4e650e7aac, role_id=b43d5a8247e244869c2c70f986f42e8a)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:33.204733 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc69a1d-112e-4eff-9a92-1287a41912fd admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:33.295622 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 975/1932] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:04:33 2022] PUT /identity/v3/projects/92b1ea6f1bef4cc7a8d0abe3e619142e/users/c50464c435524fb38747ef4e650e7aac/roles/b43d5a8247e244869c2c70f986f42e8a => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 10 13:04:33.299564 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccad23e2-26a9-44cd-9605-de7ce2b05911 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:33.314969 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccad23e2-26a9-44cd-9605-de7ce2b05911 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:33.383154 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec27c1b6-344c-49ae-8072-f4390d8a1f2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:33.383970 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec27c1b6-344c-49ae-8072-f4390d8a1f2d admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:33.384265 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec27c1b6-344c-49ae-8072-f4390d8a1f2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:33.384543 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec27c1b6-344c-49ae-8072-f4390d8a1f2d admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:33.385275 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec27c1b6-344c-49ae-8072-f4390d8a1f2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:33.387060 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec27c1b6-344c-49ae-8072-f4390d8a1f2d admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:33.400461 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 958/1933] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:04:33 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:04:33.404658 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc69a1d-112e-4eff-9a92-1287a41912fd admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:33.414791 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc69a1d-112e-4eff-9a92-1287a41912fd admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:33.419364 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3028e5-67b8-4f08-a729-f1d65400df1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:33.420294 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3028e5-67b8-4f08-a729-f1d65400df1d admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:33.420579 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3028e5-67b8-4f08-a729-f1d65400df1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:33.420855 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3028e5-67b8-4f08-a729-f1d65400df1d admin admin] PATH_INFO: `/v3/projects/92b1ea6f1bef4cc7a8d0abe3e619142e/users/c50464c435524fb38747ef4e650e7aac/roles/b43d5a8247e244869c2c70f986f42e8a` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:33.422799 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3028e5-67b8-4f08-a729-f1d65400df1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=92b1ea6f1bef4cc7a8d0abe3e619142e, user_id=c50464c435524fb38747ef4e650e7aac, role_id=b43d5a8247e244869c2c70f986f42e8a)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:33.425564 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3028e5-67b8-4f08-a729-f1d65400df1d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:33.437697 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 976/1934] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:04:33 2022] PUT /identity/v3/projects/92b1ea6f1bef4cc7a8d0abe3e619142e/users/c50464c435524fb38747ef4e650e7aac/roles/b43d5a8247e244869c2c70f986f42e8a => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:04:33.443128 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec27c1b6-344c-49ae-8072-f4390d8a1f2d admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:33.446185 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-ec27c1b6-344c-49ae-8072-f4390d8a1f2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:33.448933 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab31b2a-2829-4027-9b12-4868b44c76ab None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:33.449228 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab31b2a-2829-4027-9b12-4868b44c76ab None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:33.449505 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab31b2a-2829-4027-9b12-4868b44c76ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:33.460152 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec27c1b6-344c-49ae-8072-f4390d8a1f2d admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:33.481986 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-bab31b2a-2829-4027-9b12-4868b44c76ab None None] MFA Rules not processed for user `c50464c435524fb38747ef4e650e7aac`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:04:33.483205 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-bab31b2a-2829-4027-9b12-4868b44c76ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:33.522484 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922e91db-1b2e-469b-815e-c1c788d686b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:33.523592 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922e91db-1b2e-469b-815e-c1c788d686b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:33.523887 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922e91db-1b2e-469b-815e-c1c788d686b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:33.524173 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922e91db-1b2e-469b-815e-c1c788d686b9 admin admin] PATH_INFO: `/v3/users/83b6da423aa447f1aad0a1498f4d8f7c` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:33.525370 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922e91db-1b2e-469b-815e-c1c788d686b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83b6da423aa447f1aad0a1498f4d8f7c)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:33.527349 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922e91db-1b2e-469b-815e-c1c788d686b9 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:33.540867 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:04:33.547559 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:04:33.557089 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 977/1935] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:04:33 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:04:33.570685 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab31b2a-2829-4027-9b12-4868b44c76ab None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:33.572273 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-bab31b2a-2829-4027-9b12-4868b44c76ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:33.628477 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab31b2a-2829-4027-9b12-4868b44c76ab None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:33.632191 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562ad43f-37cc-4abd-9687-9d4db9bcd2a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6cc95a563b941ac92ab283aa0d5a5d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:33.632860 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562ad43f-37cc-4abd-9687-9d4db9bcd2a2 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:33.633139 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562ad43f-37cc-4abd-9687-9d4db9bcd2a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:33.633412 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562ad43f-37cc-4abd-9687-9d4db9bcd2a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:33.642354 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562ad43f-37cc-4abd-9687-9d4db9bcd2a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:33.643817 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562ad43f-37cc-4abd-9687-9d4db9bcd2a2 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:33.654209 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 978/1936] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:04:33 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 10 13:04:33.747914 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-922e91db-1b2e-469b-815e-c1c788d686b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83b6da423aa447f1aad0a1498f4d8f7c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:33.755218 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-922e91db-1b2e-469b-815e-c1c788d686b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83b6da423aa447f1aad0a1498f4d8f7c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:33.860225 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-922e91db-1b2e-469b-815e-c1c788d686b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83b6da423aa447f1aad0a1498f4d8f7c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:33.860847 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-922e91db-1b2e-469b-815e-c1c788d686b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83b6da423aa447f1aad0a1498f4d8f7c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:33.876391 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-922e91db-1b2e-469b-815e-c1c788d686b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83b6da423aa447f1aad0a1498f4d8f7c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:33.885595 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 959/1937] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:04:33 2022] DELETE /identity/v3/users/83b6da423aa447f1aad0a1498f4d8f7c => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:04:33.918899 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562ad43f-37cc-4abd-9687-9d4db9bcd2a2 service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:33.928643 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562ad43f-37cc-4abd-9687-9d4db9bcd2a2 service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:33.988398 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07218d3-38af-432b-b1de-b08e8d36e7e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6cc95a563b941ac92ab283aa0d5a5d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:33.989335 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07218d3-38af-432b-b1de-b08e8d36e7e5 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:33.989537 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07218d3-38af-432b-b1de-b08e8d36e7e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:33.989904 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07218d3-38af-432b-b1de-b08e8d36e7e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:33.996145 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f07218d3-38af-432b-b1de-b08e8d36e7e5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:34.020702 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922e91db-1b2e-469b-815e-c1c788d686b9 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:34.022029 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-922e91db-1b2e-469b-815e-c1c788d686b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:34.074932 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07218d3-38af-432b-b1de-b08e8d36e7e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:34.076305 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07218d3-38af-432b-b1de-b08e8d36e7e5 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:34.085282 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 979/1938] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:04:33 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:04:34.097692 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922e91db-1b2e-469b-815e-c1c788d686b9 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:34.099892 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f32f56-ba3e-4919-86f4-36b7abbb9488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:34.100776 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f32f56-ba3e-4919-86f4-36b7abbb9488 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:34.101067 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f32f56-ba3e-4919-86f4-36b7abbb9488 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:34.101893 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f32f56-ba3e-4919-86f4-36b7abbb9488 admin admin] PATH_INFO: `/v3/users/042f62a7fbf347f4b822f1edd5389d26` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:34.103114 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f32f56-ba3e-4919-86f4-36b7abbb9488 admin admin] RBAC: Authorizing `identity:delete_user(user_id=042f62a7fbf347f4b822f1edd5389d26)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:34.105094 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f32f56-ba3e-4919-86f4-36b7abbb9488 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:34.147786 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07218d3-38af-432b-b1de-b08e8d36e7e5 service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:34.149891 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f07218d3-38af-432b-b1de-b08e8d36e7e5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:34.165362 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07218d3-38af-432b-b1de-b08e8d36e7e5 service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:34.170103 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2658108f-c55a-4176-8127-f292bf3d0b82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:34.170971 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2658108f-c55a-4176-8127-f292bf3d0b82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:34.171262 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2658108f-c55a-4176-8127-f292bf3d0b82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:34.171659 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2658108f-c55a-4176-8127-f292bf3d0b82 admin admin] PATH_INFO: `/v3/projects/3835b904d7ab4046ad01c682dd49d0d7` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:34.173096 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2658108f-c55a-4176-8127-f292bf3d0b82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3835b904d7ab4046ad01c682dd49d0d7)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:34.174967 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2658108f-c55a-4176-8127-f292bf3d0b82 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:34.369192 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:04:34.369368 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c0f32f56-ba3e-4919-86f4-36b7abbb9488 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '042f62a7fbf347f4b822f1edd5389d26'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:34.372223 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2658108f-c55a-4176-8127-f292bf3d0b82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3835b904d7ab4046ad01c682dd49d0d7', 'request_id': 'req-2658108f-c55a-4176-8127-f292bf3d0b82'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:34.380826 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2658108f-c55a-4176-8127-f292bf3d0b82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3835b904d7ab4046ad01c682dd49d0d7', 'request_id': 'req-2658108f-c55a-4176-8127-f292bf3d0b82'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:34.381812 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c0f32f56-ba3e-4919-86f4-36b7abbb9488 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '042f62a7fbf347f4b822f1edd5389d26'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:34.387728 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2658108f-c55a-4176-8127-f292bf3d0b82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3835b904d7ab4046ad01c682dd49d0d7', 'request_id': 'req-2658108f-c55a-4176-8127-f292bf3d0b82'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:34.483041 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c0f32f56-ba3e-4919-86f4-36b7abbb9488 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '042f62a7fbf347f4b822f1edd5389d26'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:34.484773 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c0f32f56-ba3e-4919-86f4-36b7abbb9488 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '042f62a7fbf347f4b822f1edd5389d26'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:34.503343 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c0f32f56-ba3e-4919-86f4-36b7abbb9488 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '042f62a7fbf347f4b822f1edd5389d26'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:34.520190 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 960/1939] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:04:34 2022] DELETE /identity/v3/users/042f62a7fbf347f4b822f1edd5389d26 => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:04:34.526258 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9ed2c5-6eea-4480-ab61-ba5191915260 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:34.526655 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9ed2c5-6eea-4480-ab61-ba5191915260 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:34.526948 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9ed2c5-6eea-4480-ab61-ba5191915260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:34.529671 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2658108f-c55a-4176-8127-f292bf3d0b82 admin admin] The token cache is being invalidate because project 3835b904d7ab4046ad01c682dd49d0d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:04:34.529972 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2658108f-c55a-4176-8127-f292bf3d0b82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:34.530771 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-2658108f-c55a-4176-8127-f292bf3d0b82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:34.532804 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 980/1940] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:04:34 2022] DELETE /identity/v3/projects/3835b904d7ab4046ad01c682dd49d0d7 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 10 13:04:34.541139 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef5c6e6-a3cb-4629-8e27-84c6d0dd1d07 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:34.541436 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef5c6e6-a3cb-4629-8e27-84c6d0dd1d07 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:34.541720 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef5c6e6-a3cb-4629-8e27-84c6d0dd1d07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:34.553565 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-2e9ed2c5-6eea-4480-ab61-ba5191915260 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:04:34.554820 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-2e9ed2c5-6eea-4480-ab61-ba5191915260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:34.559016 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:04:34.560865 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-eef5c6e6-a3cb-4629-8e27-84c6d0dd1d07 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:04:34.562053 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-eef5c6e6-a3cb-4629-8e27-84c6d0dd1d07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:34.618533 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 961/1941] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:04:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 10 13:04:34.628180 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 981/1942] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:04:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 10 13:04:34.638825 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9ed2c5-6eea-4480-ab61-ba5191915260 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:34.638825 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-2e9ed2c5-6eea-4480-ab61-ba5191915260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:34.644039 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1cf043-5134-480a-a01f-0c25a9393312 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:34.644335 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1cf043-5134-480a-a01f-0c25a9393312 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:34.644615 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1cf043-5134-480a-a01f-0c25a9393312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:34.665547 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-cf1cf043-5134-480a-a01f-0c25a9393312 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:04:34.667399 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-cf1cf043-5134-480a-a01f-0c25a9393312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:34.676093 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 982/1943] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:04:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 10 13:04:34.680299 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1cf043-5134-480a-a01f-0c25a9393312 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:34.690335 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1cf043-5134-480a-a01f-0c25a9393312 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:34.691324 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9ed2c5-6eea-4480-ab61-ba5191915260 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:34.695710 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a872e9c-6ccc-4cac-b382-11f7d487b973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:34.695960 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0ea0a0-68d5-4ffd-9865-86af6d0110ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:34.696399 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0ea0a0-68d5-4ffd-9865-86af6d0110ef service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:34.696515 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a872e9c-6ccc-4cac-b382-11f7d487b973 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:34.696683 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0ea0a0-68d5-4ffd-9865-86af6d0110ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:34.696782 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a872e9c-6ccc-4cac-b382-11f7d487b973 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:34.696955 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0ea0a0-68d5-4ffd-9865-86af6d0110ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:34.697050 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a872e9c-6ccc-4cac-b382-11f7d487b973 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:34.697692 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a872e9c-6ccc-4cac-b382-11f7d487b973 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 13:04:34.698225 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a872e9c-6ccc-4cac-b382-11f7d487b973 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:34.699727 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a872e9c-6ccc-4cac-b382-11f7d487b973 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:34.708052 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0ea0a0-68d5-4ffd-9865-86af6d0110ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:34.709051 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0ea0a0-68d5-4ffd-9865-86af6d0110ef service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:34.717260 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 983/1944] 10.222.0.27 () {58 vars in 1152 bytes} [Fri Jun 10 13:04:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 13:04:34.718512 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a872e9c-6ccc-4cac-b382-11f7d487b973 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:34.722081 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 962/1945] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:04:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 10 13:04:34.729355 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a872e9c-6ccc-4cac-b382-11f7d487b973 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:34.734142 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b036e91-dfeb-4329-9edc-17d016c59e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:34.734804 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b036e91-dfeb-4329-9edc-17d016c59e90 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:34.735136 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b036e91-dfeb-4329-9edc-17d016c59e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:34.735487 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b036e91-dfeb-4329-9edc-17d016c59e90 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:34.736221 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b036e91-dfeb-4329-9edc-17d016c59e90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:34.738261 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b036e91-dfeb-4329-9edc-17d016c59e90 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:34.813839 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 984/1946] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:04:34 2022] POST /identity/v3/projects => generated 387 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 10 13:04:34.814934 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0ea0a0-68d5-4ffd-9865-86af6d0110ef service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:34.825594 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0ea0a0-68d5-4ffd-9865-86af6d0110ef service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:34.888801 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f904d3-2601-49af-91ab-8f424fa56054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:34.889475 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f904d3-2601-49af-91ab-8f424fa56054 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:34.889761 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f904d3-2601-49af-91ab-8f424fa56054 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:34.890039 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f904d3-2601-49af-91ab-8f424fa56054 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:34.890739 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f904d3-2601-49af-91ab-8f424fa56054 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:34.892696 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f904d3-2601-49af-91ab-8f424fa56054 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:34.966076 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 963/1947] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:04:34 2022] POST /identity/v3/users => generated 294 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 10 13:04:34.967771 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b036e91-dfeb-4329-9edc-17d016c59e90 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:34.979505 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b036e91-dfeb-4329-9edc-17d016c59e90 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:34.984376 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e863fa35-9209-420c-99ba-9a9e69cfa53d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:34.985137 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e863fa35-9209-420c-99ba-9a9e69cfa53d admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:34.985487 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e863fa35-9209-420c-99ba-9a9e69cfa53d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:34.985827 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e863fa35-9209-420c-99ba-9a9e69cfa53d admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:34.986621 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e863fa35-9209-420c-99ba-9a9e69cfa53d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:34.988310 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e863fa35-9209-420c-99ba-9a9e69cfa53d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:35.000971 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 985/1948] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:04:34 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 10 13:04:35.005180 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f904d3-2601-49af-91ab-8f424fa56054 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:35.016640 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f904d3-2601-49af-91ab-8f424fa56054 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:35.028433 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c405c437-6983-4c1b-92f1-5b66a0fcc197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:35.029141 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c405c437-6983-4c1b-92f1-5b66a0fcc197 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:35.029424 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c405c437-6983-4c1b-92f1-5b66a0fcc197 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:35.030039 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c405c437-6983-4c1b-92f1-5b66a0fcc197 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:35.030447 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c405c437-6983-4c1b-92f1-5b66a0fcc197 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:35.032120 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c405c437-6983-4c1b-92f1-5b66a0fcc197 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:35.042818 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 964/1949] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:04:35 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:04:35.047217 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e863fa35-9209-420c-99ba-9a9e69cfa53d admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:35.058229 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e863fa35-9209-420c-99ba-9a9e69cfa53d admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:35.063059 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b23715-edf6-44ca-affd-a0ad772bb768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:35.063998 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b23715-edf6-44ca-affd-a0ad772bb768 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:35.064281 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b23715-edf6-44ca-affd-a0ad772bb768 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:35.064557 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b23715-edf6-44ca-affd-a0ad772bb768 admin admin] PATH_INFO: `/v3/projects/c2954e97c0274c8584fcd3b70e4b7d33/users/5eaec1ec339f4e8ebf8b54243b45f6bd/roles/43106c664e1b40de984c0c62701149c8` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:35.081555 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b23715-edf6-44ca-affd-a0ad772bb768 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2954e97c0274c8584fcd3b70e4b7d33, user_id=5eaec1ec339f4e8ebf8b54243b45f6bd, role_id=43106c664e1b40de984c0c62701149c8)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:35.084222 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b23715-edf6-44ca-affd-a0ad772bb768 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:35.158334 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 986/1950] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:04:35 2022] PUT /identity/v3/projects/c2954e97c0274c8584fcd3b70e4b7d33/users/5eaec1ec339f4e8ebf8b54243b45f6bd/roles/43106c664e1b40de984c0c62701149c8 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:04:35.169187 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a76d75-02e3-45b9-8711-efb83fd4b4e6 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:35.169483 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a76d75-02e3-45b9-8711-efb83fd4b4e6 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:35.169769 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a76d75-02e3-45b9-8711-efb83fd4b4e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:35.198317 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-13a76d75-02e3-45b9-8711-efb83fd4b4e6 None None] MFA Rules not processed for user `5eaec1ec339f4e8ebf8b54243b45f6bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:04:35.199523 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-13a76d75-02e3-45b9-8711-efb83fd4b4e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:35.218383 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:04:35.266413 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 965/1951] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:04:35 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:04:35.271065 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b23715-edf6-44ca-affd-a0ad772bb768 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:35.332284 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b23715-edf6-44ca-affd-a0ad772bb768 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:35.336961 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6049371a-4d66-4ee4-9ee3-0460461e5e67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:35.337581 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6049371a-4d66-4ee4-9ee3-0460461e5e67 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:35.337867 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6049371a-4d66-4ee4-9ee3-0460461e5e67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:35.338143 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6049371a-4d66-4ee4-9ee3-0460461e5e67 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:35.338834 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6049371a-4d66-4ee4-9ee3-0460461e5e67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:35.340744 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6049371a-4d66-4ee4-9ee3-0460461e5e67 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:35.343888 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:04:35.422932 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a76d75-02e3-45b9-8711-efb83fd4b4e6 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:35.424504 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-13a76d75-02e3-45b9-8711-efb83fd4b4e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:35.440005 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a76d75-02e3-45b9-8711-efb83fd4b4e6 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:35.449017 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57293b1-71c6-482c-b6f7-b7bc6ef45fa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:35.449017 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57293b1-71c6-482c-b6f7-b7bc6ef45fa2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:35.449017 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57293b1-71c6-482c-b6f7-b7bc6ef45fa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:35.449017 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57293b1-71c6-482c-b6f7-b7bc6ef45fa2 admin admin] PATH_INFO: `/v3/projects/be33831f5c2d408f9d948d6f7ebbf674` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:35.449347 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57293b1-71c6-482c-b6f7-b7bc6ef45fa2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be33831f5c2d408f9d948d6f7ebbf674)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:35.450275 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57293b1-71c6-482c-b6f7-b7bc6ef45fa2 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:35.463471 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 987/1952] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:04:35 2022] POST /identity/v3/projects => generated 385 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 10 13:04:35.467438 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6049371a-4d66-4ee4-9ee3-0460461e5e67 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:35.532283 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6049371a-4d66-4ee4-9ee3-0460461e5e67 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:35.535253 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1d2292-3088-4667-99fc-3921a71b2898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:35.535960 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1d2292-3088-4667-99fc-3921a71b2898 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:35.536255 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1d2292-3088-4667-99fc-3921a71b2898 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:35.536507 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1d2292-3088-4667-99fc-3921a71b2898 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:35.537190 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1d2292-3088-4667-99fc-3921a71b2898 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:35.539777 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1d2292-3088-4667-99fc-3921a71b2898 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:35.589036 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f57293b1-71c6-482c-b6f7-b7bc6ef45fa2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be33831f5c2d408f9d948d6f7ebbf674', 'request_id': 'req-f57293b1-71c6-482c-b6f7-b7bc6ef45fa2'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:35.597458 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f57293b1-71c6-482c-b6f7-b7bc6ef45fa2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be33831f5c2d408f9d948d6f7ebbf674', 'request_id': 'req-f57293b1-71c6-482c-b6f7-b7bc6ef45fa2'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:35.604476 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f57293b1-71c6-482c-b6f7-b7bc6ef45fa2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be33831f5c2d408f9d948d6f7ebbf674', 'request_id': 'req-f57293b1-71c6-482c-b6f7-b7bc6ef45fa2'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:35.643693 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 988/1953] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:04:35 2022] POST /identity/v3/users => generated 299 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 10 13:04:35.649227 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1d2292-3088-4667-99fc-3921a71b2898 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:35.664983 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1d2292-3088-4667-99fc-3921a71b2898 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:35.704475 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f57293b1-71c6-482c-b6f7-b7bc6ef45fa2 admin admin] The token cache is being invalidate because project be33831f5c2d408f9d948d6f7ebbf674 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:04:35.704780 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f57293b1-71c6-482c-b6f7-b7bc6ef45fa2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:35.705552 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f57293b1-71c6-482c-b6f7-b7bc6ef45fa2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:35.707587 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 966/1954] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:04:35 2022] DELETE /identity/v3/projects/be33831f5c2d408f9d948d6f7ebbf674 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 10 13:04:35.714560 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57293b1-71c6-482c-b6f7-b7bc6ef45fa2 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:35.716230 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f57293b1-71c6-482c-b6f7-b7bc6ef45fa2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:35.744730 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a70b984-fb4f-42da-a2ac-d6699c6dd47e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:35.745403 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a70b984-fb4f-42da-a2ac-d6699c6dd47e admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:35.745685 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a70b984-fb4f-42da-a2ac-d6699c6dd47e admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:35.745993 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a70b984-fb4f-42da-a2ac-d6699c6dd47e admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:35.746718 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a70b984-fb4f-42da-a2ac-d6699c6dd47e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:35.748342 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a70b984-fb4f-42da-a2ac-d6699c6dd47e admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:35.761177 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 989/1955] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:04:35 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 10 13:04:35.775491 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a70b984-fb4f-42da-a2ac-d6699c6dd47e admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:35.777267 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-7a70b984-fb4f-42da-a2ac-d6699c6dd47e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:35.790651 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57293b1-71c6-482c-b6f7-b7bc6ef45fa2 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:35.790824 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a70b984-fb4f-42da-a2ac-d6699c6dd47e admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:35.796123 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14490a90-77d8-4813-9cb8-d25a79188107 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:35.797037 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14490a90-77d8-4813-9cb8-d25a79188107 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:35.797324 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14490a90-77d8-4813-9cb8-d25a79188107 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:35.797608 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14490a90-77d8-4813-9cb8-d25a79188107 admin admin] PATH_INFO: `/v3/projects/fdf9c2256a8c4c769c0714592c8a7a09/users/1ea84b6403194b019284d04dc491123e/roles/b43d5a8247e244869c2c70f986f42e8a` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:35.799254 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a6b639-09cd-4e56-b327-987ffba032e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:35.800238 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a6b639-09cd-4e56-b327-987ffba032e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:35.800574 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a6b639-09cd-4e56-b327-987ffba032e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:35.800862 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a6b639-09cd-4e56-b327-987ffba032e8 admin admin] PATH_INFO: `/v3/users/361442add93246b2bedbec81ab045663` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:35.802176 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a6b639-09cd-4e56-b327-987ffba032e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=361442add93246b2bedbec81ab045663)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:35.804381 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a6b639-09cd-4e56-b327-987ffba032e8 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:35.813273 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14490a90-77d8-4813-9cb8-d25a79188107 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdf9c2256a8c4c769c0714592c8a7a09, user_id=1ea84b6403194b019284d04dc491123e, role_id=b43d5a8247e244869c2c70f986f42e8a)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:35.815921 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14490a90-77d8-4813-9cb8-d25a79188107 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:35.911799 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 990/1956] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:04:35 2022] PUT /identity/v3/projects/fdf9c2256a8c4c769c0714592c8a7a09/users/1ea84b6403194b019284d04dc491123e/roles/b43d5a8247e244869c2c70f986f42e8a => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 10 13:04:35.915738 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14490a90-77d8-4813-9cb8-d25a79188107 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:35.965592 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:04:35.979034 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14490a90-77d8-4813-9cb8-d25a79188107 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:35.983969 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557f221c-1508-436c-ad05-f571f7438474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:35.984634 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557f221c-1508-436c-ad05-f571f7438474 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:35.984916 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557f221c-1508-436c-ad05-f571f7438474 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:35.985193 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557f221c-1508-436c-ad05-f571f7438474 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:35.985908 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557f221c-1508-436c-ad05-f571f7438474 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:35.987515 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557f221c-1508-436c-ad05-f571f7438474 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:36.003141 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 991/1957] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:04:35 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 10 13:04:36.004710 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557f221c-1508-436c-ad05-f571f7438474 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:36.014747 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557f221c-1508-436c-ad05-f571f7438474 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:36.019636 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94b0bd2-50d4-43a7-9948-d29c021846fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:36.020565 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94b0bd2-50d4-43a7-9948-d29c021846fe admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:36.021291 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94b0bd2-50d4-43a7-9948-d29c021846fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:36.021546 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94b0bd2-50d4-43a7-9948-d29c021846fe admin admin] PATH_INFO: `/v3/projects/fdf9c2256a8c4c769c0714592c8a7a09/users/1ea84b6403194b019284d04dc491123e/roles/b43d5a8247e244869c2c70f986f42e8a` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:36.024981 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94b0bd2-50d4-43a7-9948-d29c021846fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdf9c2256a8c4c769c0714592c8a7a09, user_id=1ea84b6403194b019284d04dc491123e, role_id=b43d5a8247e244869c2c70f986f42e8a)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:36.028101 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94b0bd2-50d4-43a7-9948-d29c021846fe admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:36.030882 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:04:36.032626 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-25a6b639-09cd-4e56-b327-987ffba032e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '361442add93246b2bedbec81ab045663'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:36.039820 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-25a6b639-09cd-4e56-b327-987ffba032e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '361442add93246b2bedbec81ab045663'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:36.053626 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 992/1958] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:04:36 2022] PUT /identity/v3/projects/fdf9c2256a8c4c769c0714592c8a7a09/users/1ea84b6403194b019284d04dc491123e/roles/b43d5a8247e244869c2c70f986f42e8a => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:04:36.064108 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbac7195-8e6d-4927-a1d3-7dbb1be016f0 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:36.064410 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbac7195-8e6d-4927-a1d3-7dbb1be016f0 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:36.064741 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbac7195-8e6d-4927-a1d3-7dbb1be016f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:36.091333 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-fbac7195-8e6d-4927-a1d3-7dbb1be016f0 None None] MFA Rules not processed for user `1ea84b6403194b019284d04dc491123e`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:04:36.092570 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-fbac7195-8e6d-4927-a1d3-7dbb1be016f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:36.156783 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-25a6b639-09cd-4e56-b327-987ffba032e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '361442add93246b2bedbec81ab045663'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:36.157527 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-25a6b639-09cd-4e56-b327-987ffba032e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '361442add93246b2bedbec81ab045663'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:36.162601 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 993/1959] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:04:36 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 10 13:04:36.174137 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-25a6b639-09cd-4e56-b327-987ffba032e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '361442add93246b2bedbec81ab045663'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:36.174600 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbac7195-8e6d-4927-a1d3-7dbb1be016f0 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:36.176023 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-fbac7195-8e6d-4927-a1d3-7dbb1be016f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:36.180555 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 967/1960] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:04:35 2022] DELETE /identity/v3/users/361442add93246b2bedbec81ab045663 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 10 13:04:36.237620 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbac7195-8e6d-4927-a1d3-7dbb1be016f0 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:36.305978 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e08d52-e594-40e3-976d-59c24152cff3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6cc95a563b941ac92ab283aa0d5a5d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:36.306487 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e08d52-e594-40e3-976d-59c24152cff3 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:36.306770 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e08d52-e594-40e3-976d-59c24152cff3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:36.307042 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e08d52-e594-40e3-976d-59c24152cff3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:36.308902 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-18e08d52-e594-40e3-976d-59c24152cff3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:36.339825 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:04:36.422512 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e08d52-e594-40e3-976d-59c24152cff3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:36.422512 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e08d52-e594-40e3-976d-59c24152cff3 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:36.440881 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 994/1961] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:04:36 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 10 13:04:36.472017 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a6b639-09cd-4e56-b327-987ffba032e8 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:36.473614 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-25a6b639-09cd-4e56-b327-987ffba032e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:36.543704 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a6b639-09cd-4e56-b327-987ffba032e8 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:36.548613 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f39f4c-d0f0-4d58-9731-57f23564829a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:36.549510 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f39f4c-d0f0-4d58-9731-57f23564829a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:36.549805 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f39f4c-d0f0-4d58-9731-57f23564829a admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:36.550127 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f39f4c-d0f0-4d58-9731-57f23564829a admin admin] PATH_INFO: `/v3/users/5eaec1ec339f4e8ebf8b54243b45f6bd` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:36.551330 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f39f4c-d0f0-4d58-9731-57f23564829a admin admin] RBAC: Authorizing `identity:delete_user(user_id=5eaec1ec339f4e8ebf8b54243b45f6bd)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:36.553283 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f39f4c-d0f0-4d58-9731-57f23564829a admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:36.593841 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e08d52-e594-40e3-976d-59c24152cff3 service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:36.595338 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-18e08d52-e594-40e3-976d-59c24152cff3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:36.611502 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e08d52-e594-40e3-976d-59c24152cff3 service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:36.617476 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bd5ce9-3e91-4c3f-8302-f811a83bc9a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:36.618349 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bd5ce9-3e91-4c3f-8302-f811a83bc9a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:36.618623 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bd5ce9-3e91-4c3f-8302-f811a83bc9a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:36.622165 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bd5ce9-3e91-4c3f-8302-f811a83bc9a6 admin admin] PATH_INFO: `/v3/projects/2e78d3d80a334b2294de1d0d6bbfbc02` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:36.622165 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bd5ce9-3e91-4c3f-8302-f811a83bc9a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e78d3d80a334b2294de1d0d6bbfbc02)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:36.622165 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bd5ce9-3e91-4c3f-8302-f811a83bc9a6 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:36.770732 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a2bd5ce9-3e91-4c3f-8302-f811a83bc9a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e78d3d80a334b2294de1d0d6bbfbc02', 'request_id': 'req-a2bd5ce9-3e91-4c3f-8302-f811a83bc9a6'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:36.774201 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-80f39f4c-d0f0-4d58-9731-57f23564829a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5eaec1ec339f4e8ebf8b54243b45f6bd'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:36.777753 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a2bd5ce9-3e91-4c3f-8302-f811a83bc9a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e78d3d80a334b2294de1d0d6bbfbc02', 'request_id': 'req-a2bd5ce9-3e91-4c3f-8302-f811a83bc9a6'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:36.783969 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-80f39f4c-d0f0-4d58-9731-57f23564829a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5eaec1ec339f4e8ebf8b54243b45f6bd'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:36.785104 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a2bd5ce9-3e91-4c3f-8302-f811a83bc9a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e78d3d80a334b2294de1d0d6bbfbc02', 'request_id': 'req-a2bd5ce9-3e91-4c3f-8302-f811a83bc9a6'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:36.860063 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-80f39f4c-d0f0-4d58-9731-57f23564829a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5eaec1ec339f4e8ebf8b54243b45f6bd'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:36.860689 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-80f39f4c-d0f0-4d58-9731-57f23564829a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5eaec1ec339f4e8ebf8b54243b45f6bd'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:36.874113 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-80f39f4c-d0f0-4d58-9731-57f23564829a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5eaec1ec339f4e8ebf8b54243b45f6bd'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:36.880086 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 968/1962] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:04:36 2022] DELETE /identity/v3/users/5eaec1ec339f4e8ebf8b54243b45f6bd => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:04:36.884326 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084fb781-af4b-4d42-b5fa-31253ade039d None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:36.884616 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084fb781-af4b-4d42-b5fa-31253ade039d None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:36.884892 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084fb781-af4b-4d42-b5fa-31253ade039d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:36.901803 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-084fb781-af4b-4d42-b5fa-31253ade039d None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:04:36.902992 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-084fb781-af4b-4d42-b5fa-31253ade039d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:36.927757 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a2bd5ce9-3e91-4c3f-8302-f811a83bc9a6 admin admin] The token cache is being invalidate because project 2e78d3d80a334b2294de1d0d6bbfbc02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:04:36.928095 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a2bd5ce9-3e91-4c3f-8302-f811a83bc9a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:36.928756 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a2bd5ce9-3e91-4c3f-8302-f811a83bc9a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:36.930875 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 995/1963] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:04:36 2022] DELETE /identity/v3/projects/2e78d3d80a334b2294de1d0d6bbfbc02 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:04:36.934700 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bd5ce9-3e91-4c3f-8302-f811a83bc9a6 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:36.936033 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a2bd5ce9-3e91-4c3f-8302-f811a83bc9a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:36.940654 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:04:36.963022 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 969/1964] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:04:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 10 13:04:36.972459 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084fb781-af4b-4d42-b5fa-31253ade039d None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:36.973803 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-084fb781-af4b-4d42-b5fa-31253ade039d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:37.004458 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bd5ce9-3e91-4c3f-8302-f811a83bc9a6 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:37.009580 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f336e00-773e-4037-8daf-c20ae0c542c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:37.010547 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f336e00-773e-4037-8daf-c20ae0c542c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:37.010833 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f336e00-773e-4037-8daf-c20ae0c542c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:37.011108 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f336e00-773e-4037-8daf-c20ae0c542c8 admin admin] PATH_INFO: `/v3/users/d81544b2fb0d47509711754805d489f7` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:37.012358 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f336e00-773e-4037-8daf-c20ae0c542c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d81544b2fb0d47509711754805d489f7)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:37.014262 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f336e00-773e-4037-8daf-c20ae0c542c8 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:37.042699 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084fb781-af4b-4d42-b5fa-31253ade039d None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:37.047017 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e84b20-31d3-47af-b258-7c15e3534d86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:37.047675 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e84b20-31d3-47af-b258-7c15e3534d86 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:37.048028 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e84b20-31d3-47af-b258-7c15e3534d86 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:37.048442 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e84b20-31d3-47af-b258-7c15e3534d86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:37.058843 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e84b20-31d3-47af-b258-7c15e3534d86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:37.060348 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e84b20-31d3-47af-b258-7c15e3534d86 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:37.070956 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 970/1965] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:04:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 10 13:04:37.307656 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:04:37.309817 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5f336e00-773e-4037-8daf-c20ae0c542c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd81544b2fb0d47509711754805d489f7'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:37.317925 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5f336e00-773e-4037-8daf-c20ae0c542c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd81544b2fb0d47509711754805d489f7'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:37.380307 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5f336e00-773e-4037-8daf-c20ae0c542c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd81544b2fb0d47509711754805d489f7'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:37.380926 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5f336e00-773e-4037-8daf-c20ae0c542c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd81544b2fb0d47509711754805d489f7'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:37.395448 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5f336e00-773e-4037-8daf-c20ae0c542c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd81544b2fb0d47509711754805d489f7'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:37.407742 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 996/1966] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:04:36 2022] DELETE /identity/v3/users/d81544b2fb0d47509711754805d489f7 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 10 13:04:37.532654 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e84b20-31d3-47af-b258-7c15e3534d86 service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:37.534411 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c2e84b20-31d3-47af-b258-7c15e3534d86 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:37.565740 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e84b20-31d3-47af-b258-7c15e3534d86 service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:37.625623 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554621e0-20c5-42f1-8db7-1bae76a9b229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:37.626505 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554621e0-20c5-42f1-8db7-1bae76a9b229 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:37.626794 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554621e0-20c5-42f1-8db7-1bae76a9b229 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:37.627072 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554621e0-20c5-42f1-8db7-1bae76a9b229 admin admin] PATH_INFO: `/v3/projects/c2954e97c0274c8584fcd3b70e4b7d33` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:37.628515 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554621e0-20c5-42f1-8db7-1bae76a9b229 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2954e97c0274c8584fcd3b70e4b7d33)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:37.630299 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554621e0-20c5-42f1-8db7-1bae76a9b229 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:37.763268 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-554621e0-20c5-42f1-8db7-1bae76a9b229 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2954e97c0274c8584fcd3b70e4b7d33', 'request_id': 'req-554621e0-20c5-42f1-8db7-1bae76a9b229'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:37.769387 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-554621e0-20c5-42f1-8db7-1bae76a9b229 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2954e97c0274c8584fcd3b70e4b7d33', 'request_id': 'req-554621e0-20c5-42f1-8db7-1bae76a9b229'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:37.777404 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-554621e0-20c5-42f1-8db7-1bae76a9b229 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2954e97c0274c8584fcd3b70e4b7d33', 'request_id': 'req-554621e0-20c5-42f1-8db7-1bae76a9b229'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:37.887236 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-554621e0-20c5-42f1-8db7-1bae76a9b229 admin admin] The token cache is being invalidate because project c2954e97c0274c8584fcd3b70e4b7d33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:04:37.887479 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-554621e0-20c5-42f1-8db7-1bae76a9b229 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:37.888220 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-554621e0-20c5-42f1-8db7-1bae76a9b229 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:37.890166 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 971/1967] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:04:37 2022] DELETE /identity/v3/projects/c2954e97c0274c8584fcd3b70e4b7d33 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:04:37.893976 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f336e00-773e-4037-8daf-c20ae0c542c8 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:37.895954 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5f336e00-773e-4037-8daf-c20ae0c542c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:37.914696 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f336e00-773e-4037-8daf-c20ae0c542c8 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:37.922225 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554621e0-20c5-42f1-8db7-1bae76a9b229 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:37.924411 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-554621e0-20c5-42f1-8db7-1bae76a9b229 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:37.961822 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:04:38.003514 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc36878c-83b1-469b-a778-d1b70cde8684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:38.004433 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc36878c-83b1-469b-a778-d1b70cde8684 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:38.010600 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc36878c-83b1-469b-a778-d1b70cde8684 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:38.010600 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc36878c-83b1-469b-a778-d1b70cde8684 admin admin] PATH_INFO: `/v3/users/1ea84b6403194b019284d04dc491123e` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:38.010600 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc36878c-83b1-469b-a778-d1b70cde8684 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ea84b6403194b019284d04dc491123e)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:38.010600 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc36878c-83b1-469b-a778-d1b70cde8684 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:38.035628 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554621e0-20c5-42f1-8db7-1bae76a9b229 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:38.048627 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b57d45-8a64-48d7-80fa-9641affd5038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:38.049546 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b57d45-8a64-48d7-80fa-9641affd5038 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:38.049827 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b57d45-8a64-48d7-80fa-9641affd5038 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:38.050099 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b57d45-8a64-48d7-80fa-9641affd5038 admin admin] PATH_INFO: `/v3/projects/946f4e6d0aa34c31a13a2f7a5e2e6d07` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:38.051298 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b57d45-8a64-48d7-80fa-9641affd5038 admin admin] RBAC: Authorizing `identity:delete_project(project_id=946f4e6d0aa34c31a13a2f7a5e2e6d07)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:38.053543 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b57d45-8a64-48d7-80fa-9641affd5038 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:38.257232 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-a8b57d45-8a64-48d7-80fa-9641affd5038 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '946f4e6d0aa34c31a13a2f7a5e2e6d07', 'request_id': 'req-a8b57d45-8a64-48d7-80fa-9641affd5038'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:38.257696 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-cc36878c-83b1-469b-a778-d1b70cde8684 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ea84b6403194b019284d04dc491123e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:38.262892 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-a8b57d45-8a64-48d7-80fa-9641affd5038 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '946f4e6d0aa34c31a13a2f7a5e2e6d07', 'request_id': 'req-a8b57d45-8a64-48d7-80fa-9641affd5038'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:38.266012 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-cc36878c-83b1-469b-a778-d1b70cde8684 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ea84b6403194b019284d04dc491123e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:38.270696 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-a8b57d45-8a64-48d7-80fa-9641affd5038 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '946f4e6d0aa34c31a13a2f7a5e2e6d07', 'request_id': 'req-a8b57d45-8a64-48d7-80fa-9641affd5038'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:38.363008 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-cc36878c-83b1-469b-a778-d1b70cde8684 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ea84b6403194b019284d04dc491123e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:38.363666 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-cc36878c-83b1-469b-a778-d1b70cde8684 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ea84b6403194b019284d04dc491123e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:38.377065 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-cc36878c-83b1-469b-a778-d1b70cde8684 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ea84b6403194b019284d04dc491123e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:38.383290 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 997/1968] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:04:37 2022] DELETE /identity/v3/users/1ea84b6403194b019284d04dc491123e => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 10 13:04:38.425504 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-a8b57d45-8a64-48d7-80fa-9641affd5038 admin admin] The token cache is being invalidate because project 946f4e6d0aa34c31a13a2f7a5e2e6d07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:04:38.426455 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-a8b57d45-8a64-48d7-80fa-9641affd5038 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:38.427474 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-a8b57d45-8a64-48d7-80fa-9641affd5038 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:38.434845 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 972/1969] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:04:37 2022] DELETE /identity/v3/projects/946f4e6d0aa34c31a13a2f7a5e2e6d07 => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:04:38.436062 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110e323d-3639-4327-ba3f-e35a7b443c9d None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:38.436356 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110e323d-3639-4327-ba3f-e35a7b443c9d None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:38.436636 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110e323d-3639-4327-ba3f-e35a7b443c9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:38.454294 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-110e323d-3639-4327-ba3f-e35a7b443c9d None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:04:38.456514 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-110e323d-3639-4327-ba3f-e35a7b443c9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:38.501612 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:04:38.522670 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 998/1970] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:04:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 10 13:04:38.532735 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1ea4fe-3aa2-4a53-9eea-5f3349320c1c None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:38.533290 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1ea4fe-3aa2-4a53-9eea-5f3349320c1c None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:38.533596 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1ea4fe-3aa2-4a53-9eea-5f3349320c1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:38.552862 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-9c1ea4fe-3aa2-4a53-9eea-5f3349320c1c None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:04:38.553997 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-9c1ea4fe-3aa2-4a53-9eea-5f3349320c1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:38.562227 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 973/1971] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:04:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:04:38.566566 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110e323d-3639-4327-ba3f-e35a7b443c9d None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:38.576910 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110e323d-3639-4327-ba3f-e35a7b443c9d None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:38.581594 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8478a59-1a39-4c50-984d-b95888af5267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:38.582333 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8478a59-1a39-4c50-984d-b95888af5267 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:38.582624 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8478a59-1a39-4c50-984d-b95888af5267 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:38.582896 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8478a59-1a39-4c50-984d-b95888af5267 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:38.583645 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8478a59-1a39-4c50-984d-b95888af5267 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 13:04:38.583991 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8478a59-1a39-4c50-984d-b95888af5267 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:38.586256 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8478a59-1a39-4c50-984d-b95888af5267 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:38.608302 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 999/1972] 10.222.0.27 () {58 vars in 1152 bytes} [Fri Jun 10 13:04:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 13:04:38.613660 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1ea4fe-3aa2-4a53-9eea-5f3349320c1c None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:38.636235 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1ea4fe-3aa2-4a53-9eea-5f3349320c1c None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:38.637831 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49665c02-0e3a-4f5b-8e15-e3ac7cc77390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:38.638610 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49665c02-0e3a-4f5b-8e15-e3ac7cc77390 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:38.641736 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49665c02-0e3a-4f5b-8e15-e3ac7cc77390 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:38.641736 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49665c02-0e3a-4f5b-8e15-e3ac7cc77390 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:38.641736 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49665c02-0e3a-4f5b-8e15-e3ac7cc77390 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:38.642394 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49665c02-0e3a-4f5b-8e15-e3ac7cc77390 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:38.757311 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 974/1973] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:04:38 2022] POST /identity/v3/projects => generated 383 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:04:38.761440 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8478a59-1a39-4c50-984d-b95888af5267 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:38.771591 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8478a59-1a39-4c50-984d-b95888af5267 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:38.828013 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b900c845-290b-40e2-aedd-f4ca692bc45a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:38.828604 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b900c845-290b-40e2-aedd-f4ca692bc45a admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:38.828890 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b900c845-290b-40e2-aedd-f4ca692bc45a admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:38.829168 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b900c845-290b-40e2-aedd-f4ca692bc45a admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:38.829862 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b900c845-290b-40e2-aedd-f4ca692bc45a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:38.831794 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b900c845-290b-40e2-aedd-f4ca692bc45a admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:38.873176 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49665c02-0e3a-4f5b-8e15-e3ac7cc77390 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:38.874998 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-49665c02-0e3a-4f5b-8e15-e3ac7cc77390 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:38.889245 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49665c02-0e3a-4f5b-8e15-e3ac7cc77390 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:38.897729 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a30872b-900b-4e67-a907-a63b97e5ea99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:38.899196 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a30872b-900b-4e67-a907-a63b97e5ea99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:38.899748 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a30872b-900b-4e67-a907-a63b97e5ea99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:38.900060 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a30872b-900b-4e67-a907-a63b97e5ea99 admin admin] PATH_INFO: `/v3/projects/fdf9c2256a8c4c769c0714592c8a7a09` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:38.904802 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a30872b-900b-4e67-a907-a63b97e5ea99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fdf9c2256a8c4c769c0714592c8a7a09)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:38.906817 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a30872b-900b-4e67-a907-a63b97e5ea99 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:38.929341 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1000/1974] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:04:38 2022] POST /identity/v3/users => generated 292 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:04:38.937993 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b900c845-290b-40e2-aedd-f4ca692bc45a admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:38.948573 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b900c845-290b-40e2-aedd-f4ca692bc45a admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:38.953316 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02a79ed-09e8-4802-8413-7967ccd776be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:38.953970 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02a79ed-09e8-4802-8413-7967ccd776be admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:38.954253 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02a79ed-09e8-4802-8413-7967ccd776be admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:38.954531 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02a79ed-09e8-4802-8413-7967ccd776be admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:38.955253 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02a79ed-09e8-4802-8413-7967ccd776be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:38.956890 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02a79ed-09e8-4802-8413-7967ccd776be admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:38.967306 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1001/1975] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:04:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 10 13:04:38.972405 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02a79ed-09e8-4802-8413-7967ccd776be admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:38.982625 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02a79ed-09e8-4802-8413-7967ccd776be admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:38.987428 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c374955e-66c7-4659-b34b-e3dd94c8bb59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:38.988148 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c374955e-66c7-4659-b34b-e3dd94c8bb59 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:38.988456 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c374955e-66c7-4659-b34b-e3dd94c8bb59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:38.988736 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c374955e-66c7-4659-b34b-e3dd94c8bb59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:38.989452 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c374955e-66c7-4659-b34b-e3dd94c8bb59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:38.991111 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c374955e-66c7-4659-b34b-e3dd94c8bb59 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:39.001937 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1002/1976] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:04:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:04:39.005867 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c374955e-66c7-4659-b34b-e3dd94c8bb59 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:39.015417 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c374955e-66c7-4659-b34b-e3dd94c8bb59 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:39.020011 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba820428-2646-4e08-afe6-593007b9babe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:39.020842 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba820428-2646-4e08-afe6-593007b9babe admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:39.021123 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba820428-2646-4e08-afe6-593007b9babe admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:39.021397 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba820428-2646-4e08-afe6-593007b9babe admin admin] PATH_INFO: `/v3/projects/b6d1773e054c476b941c5661bc1fad5b/users/5b58c9a7fcea4e75920923e3caba48b9/roles/43106c664e1b40de984c0c62701149c8` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:39.036289 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba820428-2646-4e08-afe6-593007b9babe admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6d1773e054c476b941c5661bc1fad5b, user_id=5b58c9a7fcea4e75920923e3caba48b9, role_id=43106c664e1b40de984c0c62701149c8)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:39.039143 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba820428-2646-4e08-afe6-593007b9babe admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:39.056768 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4a30872b-900b-4e67-a907-a63b97e5ea99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fdf9c2256a8c4c769c0714592c8a7a09', 'request_id': 'req-4a30872b-900b-4e67-a907-a63b97e5ea99'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:39.062198 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4a30872b-900b-4e67-a907-a63b97e5ea99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fdf9c2256a8c4c769c0714592c8a7a09', 'request_id': 'req-4a30872b-900b-4e67-a907-a63b97e5ea99'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:39.068901 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4a30872b-900b-4e67-a907-a63b97e5ea99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fdf9c2256a8c4c769c0714592c8a7a09', 'request_id': 'req-4a30872b-900b-4e67-a907-a63b97e5ea99'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:39.141284 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1003/1977] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:04:39 2022] PUT /identity/v3/projects/b6d1773e054c476b941c5661bc1fad5b/users/5b58c9a7fcea4e75920923e3caba48b9/roles/43106c664e1b40de984c0c62701149c8 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 10 13:04:39.152921 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431d09b5-9385-4f52-87bf-e32ffcadf0ff None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:39.153217 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431d09b5-9385-4f52-87bf-e32ffcadf0ff None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:39.153490 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431d09b5-9385-4f52-87bf-e32ffcadf0ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:39.170743 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:04:39.176137 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4a30872b-900b-4e67-a907-a63b97e5ea99 admin admin] The token cache is being invalidate because project fdf9c2256a8c4c769c0714592c8a7a09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:04:39.182336 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4a30872b-900b-4e67-a907-a63b97e5ea99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:39.182336 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4a30872b-900b-4e67-a907-a63b97e5ea99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:39.182336 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 975/1978] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:04:38 2022] DELETE /identity/v3/projects/fdf9c2256a8c4c769c0714592c8a7a09 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 10 13:04:39.184078 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21768b35-904c-4072-a298-1c5d0691f55d None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:39.184428 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21768b35-904c-4072-a298-1c5d0691f55d None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:39.184774 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21768b35-904c-4072-a298-1c5d0691f55d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:39.195404 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-431d09b5-9385-4f52-87bf-e32ffcadf0ff None None] MFA Rules not processed for user `5b58c9a7fcea4e75920923e3caba48b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:04:39.197174 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-431d09b5-9385-4f52-87bf-e32ffcadf0ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:39.219958 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-21768b35-904c-4072-a298-1c5d0691f55d None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:04:39.221220 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-21768b35-904c-4072-a298-1c5d0691f55d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:39.283950 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1004/1979] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:04:39 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 10 13:04:39.291792 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 976/1980] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:04:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 10 13:04:39.297519 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431d09b5-9385-4f52-87bf-e32ffcadf0ff None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:39.298885 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-431d09b5-9385-4f52-87bf-e32ffcadf0ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:39.299287 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f53c226-e016-4cdd-935d-5df466ac2eee None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:39.299581 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f53c226-e016-4cdd-935d-5df466ac2eee None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:39.299889 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f53c226-e016-4cdd-935d-5df466ac2eee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:39.304810 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:04:39.317973 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-2f53c226-e016-4cdd-935d-5df466ac2eee None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:04:39.319233 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-2f53c226-e016-4cdd-935d-5df466ac2eee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:39.330453 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 977/1981] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:04:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 10 13:04:39.346305 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f53c226-e016-4cdd-935d-5df466ac2eee None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:39.355435 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f53c226-e016-4cdd-935d-5df466ac2eee None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:39.358779 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431d09b5-9385-4f52-87bf-e32ffcadf0ff None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:39.360173 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dc2835-53d8-46dc-89c2-1530bf707cd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:39.360830 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dc2835-53d8-46dc-89c2-1530bf707cd3 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:39.361119 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dc2835-53d8-46dc-89c2-1530bf707cd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:39.361441 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dc2835-53d8-46dc-89c2-1530bf707cd3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:39.362147 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dc2835-53d8-46dc-89c2-1530bf707cd3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 13:04:39.362488 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dc2835-53d8-46dc-89c2-1530bf707cd3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:39.363137 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162c76da-8c10-47f8-80ca-bcb4668d413d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6cc95a563b941ac92ab283aa0d5a5d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:39.363801 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162c76da-8c10-47f8-80ca-bcb4668d413d service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:39.364064 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dc2835-53d8-46dc-89c2-1530bf707cd3 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:39.364160 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162c76da-8c10-47f8-80ca-bcb4668d413d service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:39.364605 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162c76da-8c10-47f8-80ca-bcb4668d413d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:39.375348 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162c76da-8c10-47f8-80ca-bcb4668d413d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:39.376936 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 978/1982] 10.222.0.27 () {58 vars in 1152 bytes} [Fri Jun 10 13:04:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 13:04:39.377057 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162c76da-8c10-47f8-80ca-bcb4668d413d service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:39.381127 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dc2835-53d8-46dc-89c2-1530bf707cd3 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:39.386889 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1005/1983] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:04:39 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 10 13:04:39.391315 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dc2835-53d8-46dc-89c2-1530bf707cd3 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:39.396001 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fd814b-930b-4ac3-aace-fff9e6a29c2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:39.398349 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fd814b-930b-4ac3-aace-fff9e6a29c2d admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:39.398349 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fd814b-930b-4ac3-aace-fff9e6a29c2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:39.398349 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fd814b-930b-4ac3-aace-fff9e6a29c2d admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:39.398349 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fd814b-930b-4ac3-aace-fff9e6a29c2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:39.400127 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fd814b-930b-4ac3-aace-fff9e6a29c2d admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:39.502701 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 979/1984] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:04:39 2022] POST /identity/v3/projects => generated 391 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 10 13:04:39.507928 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162c76da-8c10-47f8-80ca-bcb4668d413d service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:39.518541 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162c76da-8c10-47f8-80ca-bcb4668d413d service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:39.588580 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6ebf56-527e-45c3-a6b4-385005399704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:39.589379 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6ebf56-527e-45c3-a6b4-385005399704 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:39.589664 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6ebf56-527e-45c3-a6b4-385005399704 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:39.589985 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6ebf56-527e-45c3-a6b4-385005399704 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:39.590691 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6ebf56-527e-45c3-a6b4-385005399704 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:39.592632 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6ebf56-527e-45c3-a6b4-385005399704 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:39.693154 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1006/1985] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:04:39 2022] POST /identity/v3/users => generated 296 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 10 13:04:39.699353 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fd814b-930b-4ac3-aace-fff9e6a29c2d admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:39.709891 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fd814b-930b-4ac3-aace-fff9e6a29c2d admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:39.715193 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f28706-30b1-4e44-8ed1-bc479529b015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:39.715911 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f28706-30b1-4e44-8ed1-bc479529b015 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:39.716558 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f28706-30b1-4e44-8ed1-bc479529b015 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:39.717017 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f28706-30b1-4e44-8ed1-bc479529b015 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:39.717802 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f28706-30b1-4e44-8ed1-bc479529b015 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:39.720049 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f28706-30b1-4e44-8ed1-bc479529b015 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:39.733044 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 980/1986] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:04:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:04:39.734957 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6ebf56-527e-45c3-a6b4-385005399704 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:39.745129 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6ebf56-527e-45c3-a6b4-385005399704 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:39.749987 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d1b0ec-039e-4460-94c5-7fb85acf4962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:39.750652 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d1b0ec-039e-4460-94c5-7fb85acf4962 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:39.750928 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d1b0ec-039e-4460-94c5-7fb85acf4962 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:39.751210 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d1b0ec-039e-4460-94c5-7fb85acf4962 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:39.752280 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d1b0ec-039e-4460-94c5-7fb85acf4962 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:39.754802 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d1b0ec-039e-4460-94c5-7fb85acf4962 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:39.770430 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1007/1987] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:04:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:04:39.771660 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f28706-30b1-4e44-8ed1-bc479529b015 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:39.782961 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f28706-30b1-4e44-8ed1-bc479529b015 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:39.787710 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17c5637-2579-4312-87bb-3a75377db681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:39.788710 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17c5637-2579-4312-87bb-3a75377db681 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:39.788997 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17c5637-2579-4312-87bb-3a75377db681 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:39.789305 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17c5637-2579-4312-87bb-3a75377db681 admin admin] PATH_INFO: `/v3/projects/3dfbc1605a404a10a4f6428ee51032e4/users/116b4bdc30ae4054947ace2a82d01b01/roles/43106c664e1b40de984c0c62701149c8` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:39.804908 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17c5637-2579-4312-87bb-3a75377db681 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3dfbc1605a404a10a4f6428ee51032e4, user_id=116b4bdc30ae4054947ace2a82d01b01, role_id=43106c664e1b40de984c0c62701149c8)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:39.807958 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17c5637-2579-4312-87bb-3a75377db681 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:39.893319 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 981/1988] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:04:39 2022] PUT /identity/v3/projects/3dfbc1605a404a10a4f6428ee51032e4/users/116b4bdc30ae4054947ace2a82d01b01/roles/43106c664e1b40de984c0c62701149c8 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:04:39.905611 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680418f5-15ce-4413-aa1b-50ceab3bc607 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:39.905918 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680418f5-15ce-4413-aa1b-50ceab3bc607 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:39.906230 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680418f5-15ce-4413-aa1b-50ceab3bc607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:39.935879 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-680418f5-15ce-4413-aa1b-50ceab3bc607 None None] MFA Rules not processed for user `116b4bdc30ae4054947ace2a82d01b01`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:04:39.937275 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-680418f5-15ce-4413-aa1b-50ceab3bc607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:40.009171 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1008/1989] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:04:39 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:04:40.014100 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17c5637-2579-4312-87bb-3a75377db681 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:40.094742 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17c5637-2579-4312-87bb-3a75377db681 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:40.099267 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7932a988-3973-465d-9ccd-4ea0ca07c350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:40.099989 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7932a988-3973-465d-9ccd-4ea0ca07c350 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:40.100279 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7932a988-3973-465d-9ccd-4ea0ca07c350 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:40.100555 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7932a988-3973-465d-9ccd-4ea0ca07c350 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:40.101310 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7932a988-3973-465d-9ccd-4ea0ca07c350 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:40.103571 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7932a988-3973-465d-9ccd-4ea0ca07c350 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:40.223420 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 982/1990] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:04:40 2022] POST /identity/v3/projects => generated 393 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:04:40.227538 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680418f5-15ce-4413-aa1b-50ceab3bc607 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:40.238605 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680418f5-15ce-4413-aa1b-50ceab3bc607 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:40.248412 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:04:40.301799 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbba805-5acb-4ed4-8ee6-18677005f311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:40.302580 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbba805-5acb-4ed4-8ee6-18677005f311 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:40.302961 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbba805-5acb-4ed4-8ee6-18677005f311 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:40.303356 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbba805-5acb-4ed4-8ee6-18677005f311 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:40.305026 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbba805-5acb-4ed4-8ee6-18677005f311 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:40.307032 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbba805-5acb-4ed4-8ee6-18677005f311 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:40.406945 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1009/1991] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:04:40 2022] POST /identity/v3/users => generated 303 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 10 13:04:40.411089 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7932a988-3973-465d-9ccd-4ea0ca07c350 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:40.422208 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7932a988-3973-465d-9ccd-4ea0ca07c350 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:40.426782 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90b4f9f-1a0b-4b16-b85d-d0f012b3abf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:40.427445 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90b4f9f-1a0b-4b16-b85d-d0f012b3abf8 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:40.427728 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90b4f9f-1a0b-4b16-b85d-d0f012b3abf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:40.428094 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90b4f9f-1a0b-4b16-b85d-d0f012b3abf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:40.429632 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90b4f9f-1a0b-4b16-b85d-d0f012b3abf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:40.431226 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90b4f9f-1a0b-4b16-b85d-d0f012b3abf8 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:40.445242 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:04:40.445242 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 983/1992] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:04:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 10 13:04:40.448659 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbba805-5acb-4ed4-8ee6-18677005f311 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:40.459010 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbba805-5acb-4ed4-8ee6-18677005f311 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:40.464052 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6730a295-5668-49c7-a573-c455d03fa169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:40.464909 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6730a295-5668-49c7-a573-c455d03fa169 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:40.465191 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6730a295-5668-49c7-a573-c455d03fa169 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:40.465466 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6730a295-5668-49c7-a573-c455d03fa169 admin admin] PATH_INFO: `/v3/projects/bcee24bbd42540c692f0fb14e8d39b79/users/196fcd8baa3e4468b2a321119c1224f0/roles/b43d5a8247e244869c2c70f986f42e8a` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:40.480996 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6730a295-5668-49c7-a573-c455d03fa169 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcee24bbd42540c692f0fb14e8d39b79, user_id=196fcd8baa3e4468b2a321119c1224f0, role_id=b43d5a8247e244869c2c70f986f42e8a)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:40.483574 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6730a295-5668-49c7-a573-c455d03fa169 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:40.590156 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1010/1993] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:04:40 2022] PUT /identity/v3/projects/bcee24bbd42540c692f0fb14e8d39b79/users/196fcd8baa3e4468b2a321119c1224f0/roles/b43d5a8247e244869c2c70f986f42e8a => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:04:40.594096 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90b4f9f-1a0b-4b16-b85d-d0f012b3abf8 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:40.604616 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90b4f9f-1a0b-4b16-b85d-d0f012b3abf8 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:40.665010 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95c8c26-22d0-4e33-9e6d-0843ae2ba18c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:40.665676 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95c8c26-22d0-4e33-9e6d-0843ae2ba18c admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:40.665961 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95c8c26-22d0-4e33-9e6d-0843ae2ba18c admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:40.666297 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95c8c26-22d0-4e33-9e6d-0843ae2ba18c admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:40.667015 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95c8c26-22d0-4e33-9e6d-0843ae2ba18c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:40.668641 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95c8c26-22d0-4e33-9e6d-0843ae2ba18c admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:40.678804 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 984/1994] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:04:40 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:04:40.683345 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6730a295-5668-49c7-a573-c455d03fa169 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:40.693773 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6730a295-5668-49c7-a573-c455d03fa169 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:40.699033 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ea9c49-d0cc-4467-970c-1733bf9188e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:40.699948 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ea9c49-d0cc-4467-970c-1733bf9188e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:40.703751 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ea9c49-d0cc-4467-970c-1733bf9188e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:40.703751 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ea9c49-d0cc-4467-970c-1733bf9188e2 admin admin] PATH_INFO: `/v3/projects/bcee24bbd42540c692f0fb14e8d39b79/users/196fcd8baa3e4468b2a321119c1224f0/roles/b43d5a8247e244869c2c70f986f42e8a` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:40.703751 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ea9c49-d0cc-4467-970c-1733bf9188e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcee24bbd42540c692f0fb14e8d39b79, user_id=196fcd8baa3e4468b2a321119c1224f0, role_id=b43d5a8247e244869c2c70f986f42e8a)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:40.705066 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ea9c49-d0cc-4467-970c-1733bf9188e2 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:40.715776 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1011/1995] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:04:40 2022] PUT /identity/v3/projects/bcee24bbd42540c692f0fb14e8d39b79/users/196fcd8baa3e4468b2a321119c1224f0/roles/b43d5a8247e244869c2c70f986f42e8a => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 10 13:04:40.725790 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bb4aff-e1fa-4a92-8427-95b72d60c72a None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:40.726086 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bb4aff-e1fa-4a92-8427-95b72d60c72a None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:40.726371 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bb4aff-e1fa-4a92-8427-95b72d60c72a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:40.752898 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-b2bb4aff-e1fa-4a92-8427-95b72d60c72a None None] MFA Rules not processed for user `196fcd8baa3e4468b2a321119c1224f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:04:40.754099 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b2bb4aff-e1fa-4a92-8427-95b72d60c72a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:40.824218 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 985/1996] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:04:40 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:04:40.834945 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ea9c49-d0cc-4467-970c-1733bf9188e2 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:40.836669 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-36ea9c49-d0cc-4467-970c-1733bf9188e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:40.896263 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ea9c49-d0cc-4467-970c-1733bf9188e2 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:40.900304 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bf7559-3710-491d-89d7-31dd44112605 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6cc95a563b941ac92ab283aa0d5a5d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:40.900943 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bf7559-3710-491d-89d7-31dd44112605 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:40.901224 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bf7559-3710-491d-89d7-31dd44112605 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:40.901500 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bf7559-3710-491d-89d7-31dd44112605 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:40.910349 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bf7559-3710-491d-89d7-31dd44112605 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:40.911743 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bf7559-3710-491d-89d7-31dd44112605 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:40.920975 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1012/1997] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:04:40 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:04:40.968916 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bb4aff-e1fa-4a92-8427-95b72d60c72a None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:40.978404 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bb4aff-e1fa-4a92-8427-95b72d60c72a None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:40.981932 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa70df7-d98a-4ebf-b149-2b698935f4e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6cc95a563b941ac92ab283aa0d5a5d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:40.982566 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa70df7-d98a-4ebf-b149-2b698935f4e8 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:40.982847 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa70df7-d98a-4ebf-b149-2b698935f4e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:40.983125 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa70df7-d98a-4ebf-b149-2b698935f4e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:41.041048 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa70df7-d98a-4ebf-b149-2b698935f4e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:41.042412 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa70df7-d98a-4ebf-b149-2b698935f4e8 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:41.051257 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 986/1998] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:04:40 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:04:44.172755 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bf7559-3710-491d-89d7-31dd44112605 service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:44.174136 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e7bf7559-3710-491d-89d7-31dd44112605 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:44.246649 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bf7559-3710-491d-89d7-31dd44112605 service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:44.251239 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0442299c-ed99-43cb-ae8a-d1df5725bfc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:44.252128 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0442299c-ed99-43cb-ae8a-d1df5725bfc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:44.252410 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0442299c-ed99-43cb-ae8a-d1df5725bfc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:44.252681 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0442299c-ed99-43cb-ae8a-d1df5725bfc0 admin admin] PATH_INFO: `/v3/users/5dc2ece2db0d460d99f10bf695982335` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:44.253857 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0442299c-ed99-43cb-ae8a-d1df5725bfc0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5dc2ece2db0d460d99f10bf695982335)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:44.255755 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0442299c-ed99-43cb-ae8a-d1df5725bfc0 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:44.267070 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:04:44.411631 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0442299c-ed99-43cb-ae8a-d1df5725bfc0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5dc2ece2db0d460d99f10bf695982335'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:44.419181 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0442299c-ed99-43cb-ae8a-d1df5725bfc0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5dc2ece2db0d460d99f10bf695982335'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:44.482825 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0442299c-ed99-43cb-ae8a-d1df5725bfc0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5dc2ece2db0d460d99f10bf695982335'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:44.483572 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0442299c-ed99-43cb-ae8a-d1df5725bfc0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5dc2ece2db0d460d99f10bf695982335'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:44.497468 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0442299c-ed99-43cb-ae8a-d1df5725bfc0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5dc2ece2db0d460d99f10bf695982335'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:44.506339 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1013/1999] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:04:44 2022] DELETE /identity/v3/users/5dc2ece2db0d460d99f10bf695982335 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:04:44.512336 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0fc552-604e-4153-a57e-29d3f63f834b None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:44.512823 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0fc552-604e-4153-a57e-29d3f63f834b None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:44.513143 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0fc552-604e-4153-a57e-29d3f63f834b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:44.531300 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-4a0fc552-604e-4153-a57e-29d3f63f834b None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:04:44.532745 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-4a0fc552-604e-4153-a57e-29d3f63f834b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:44.545733 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:04:44.595075 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 987/2000] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:04:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:04:44.604447 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0442299c-ed99-43cb-ae8a-d1df5725bfc0 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:44.605839 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0442299c-ed99-43cb-ae8a-d1df5725bfc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:44.667048 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0442299c-ed99-43cb-ae8a-d1df5725bfc0 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:44.670924 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6551ae-ec7f-4b59-b841-9b2a35369308 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:44.671577 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6551ae-ec7f-4b59-b841-9b2a35369308 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:44.671886 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6551ae-ec7f-4b59-b841-9b2a35369308 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:44.672197 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6551ae-ec7f-4b59-b841-9b2a35369308 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:44.684423 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6551ae-ec7f-4b59-b841-9b2a35369308 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:44.687548 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6551ae-ec7f-4b59-b841-9b2a35369308 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:44.696024 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1014/2001] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:04:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:04:45.142765 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0fc552-604e-4153-a57e-29d3f63f834b None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:45.144624 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-4a0fc552-604e-4153-a57e-29d3f63f834b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:45.161158 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0fc552-604e-4153-a57e-29d3f63f834b None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:45.166215 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad40189-c764-45ab-b95e-2fd89b00a038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:45.167580 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad40189-c764-45ab-b95e-2fd89b00a038 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:45.167852 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad40189-c764-45ab-b95e-2fd89b00a038 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:45.168171 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad40189-c764-45ab-b95e-2fd89b00a038 admin admin] PATH_INFO: `/v3/projects/9c270c03baff4b448da6253c05a1e295` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:45.169964 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad40189-c764-45ab-b95e-2fd89b00a038 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c270c03baff4b448da6253c05a1e295)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:45.171907 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad40189-c764-45ab-b95e-2fd89b00a038 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:45.307309 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-9ad40189-c764-45ab-b95e-2fd89b00a038 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c270c03baff4b448da6253c05a1e295', 'request_id': 'req-9ad40189-c764-45ab-b95e-2fd89b00a038'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:45.313125 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-9ad40189-c764-45ab-b95e-2fd89b00a038 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c270c03baff4b448da6253c05a1e295', 'request_id': 'req-9ad40189-c764-45ab-b95e-2fd89b00a038'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:45.319671 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-9ad40189-c764-45ab-b95e-2fd89b00a038 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c270c03baff4b448da6253c05a1e295', 'request_id': 'req-9ad40189-c764-45ab-b95e-2fd89b00a038'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:45.384498 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-9ad40189-c764-45ab-b95e-2fd89b00a038 admin admin] The token cache is being invalidate because project 9c270c03baff4b448da6253c05a1e295 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:04:45.384921 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-9ad40189-c764-45ab-b95e-2fd89b00a038 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:45.385554 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-9ad40189-c764-45ab-b95e-2fd89b00a038 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:45.387106 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 988/2002] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:04:45 2022] DELETE /identity/v3/projects/9c270c03baff4b448da6253c05a1e295 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:04:45.391063 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6551ae-ec7f-4b59-b841-9b2a35369308 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:45.401464 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6551ae-ec7f-4b59-b841-9b2a35369308 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:45.460099 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a84853f-9383-46a7-91cf-fc43a6109a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:45.460975 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a84853f-9383-46a7-91cf-fc43a6109a53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:45.461314 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a84853f-9383-46a7-91cf-fc43a6109a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:45.461595 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a84853f-9383-46a7-91cf-fc43a6109a53 admin admin] PATH_INFO: `/v3/users/c50464c435524fb38747ef4e650e7aac` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:45.462818 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a84853f-9383-46a7-91cf-fc43a6109a53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c50464c435524fb38747ef4e650e7aac)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:45.465288 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a84853f-9383-46a7-91cf-fc43a6109a53 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:45.642871 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:04:45.656090 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-9a84853f-9383-46a7-91cf-fc43a6109a53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c50464c435524fb38747ef4e650e7aac'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:45.663290 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-9a84853f-9383-46a7-91cf-fc43a6109a53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c50464c435524fb38747ef4e650e7aac'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:45.717242 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-9a84853f-9383-46a7-91cf-fc43a6109a53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c50464c435524fb38747ef4e650e7aac'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:45.717959 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-9a84853f-9383-46a7-91cf-fc43a6109a53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c50464c435524fb38747ef4e650e7aac'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:45.729948 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-9a84853f-9383-46a7-91cf-fc43a6109a53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c50464c435524fb38747ef4e650e7aac'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:45.735711 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1015/2003] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:04:45 2022] DELETE /identity/v3/users/c50464c435524fb38747ef4e650e7aac => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:04:46.146979 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad40189-c764-45ab-b95e-2fd89b00a038 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:46.148620 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-9ad40189-c764-45ab-b95e-2fd89b00a038 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:46.167264 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad40189-c764-45ab-b95e-2fd89b00a038 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:46.230100 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce240a1d-20eb-4984-9fd9-7a74e468bf41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:46.230970 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce240a1d-20eb-4984-9fd9-7a74e468bf41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:46.231258 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce240a1d-20eb-4984-9fd9-7a74e468bf41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:46.231539 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce240a1d-20eb-4984-9fd9-7a74e468bf41 admin admin] PATH_INFO: `/v3/projects/92b1ea6f1bef4cc7a8d0abe3e619142e` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:46.232821 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce240a1d-20eb-4984-9fd9-7a74e468bf41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=92b1ea6f1bef4cc7a8d0abe3e619142e)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:46.235473 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce240a1d-20eb-4984-9fd9-7a74e468bf41 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:46.331181 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:04:46.334881 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ce240a1d-20eb-4984-9fd9-7a74e468bf41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92b1ea6f1bef4cc7a8d0abe3e619142e', 'request_id': 'req-ce240a1d-20eb-4984-9fd9-7a74e468bf41'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:46.340146 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ce240a1d-20eb-4984-9fd9-7a74e468bf41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92b1ea6f1bef4cc7a8d0abe3e619142e', 'request_id': 'req-ce240a1d-20eb-4984-9fd9-7a74e468bf41'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:46.346261 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ce240a1d-20eb-4984-9fd9-7a74e468bf41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92b1ea6f1bef4cc7a8d0abe3e619142e', 'request_id': 'req-ce240a1d-20eb-4984-9fd9-7a74e468bf41'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:46.396009 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ce240a1d-20eb-4984-9fd9-7a74e468bf41 admin admin] The token cache is being invalidate because project 92b1ea6f1bef4cc7a8d0abe3e619142e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:04:46.396304 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ce240a1d-20eb-4984-9fd9-7a74e468bf41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:46.396921 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ce240a1d-20eb-4984-9fd9-7a74e468bf41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:46.398847 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 989/2004] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:04:46 2022] DELETE /identity/v3/projects/92b1ea6f1bef4cc7a8d0abe3e619142e => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:04:46.403905 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed68c71-4595-4838-b9b9-b6798e476749 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:46.404386 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed68c71-4595-4838-b9b9-b6798e476749 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:46.404481 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed68c71-4595-4838-b9b9-b6798e476749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:46.423695 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-0ed68c71-4595-4838-b9b9-b6798e476749 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:04:46.424902 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0ed68c71-4595-4838-b9b9-b6798e476749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:46.483406 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1016/2005] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:04:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:04:46.491032 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e875036f-2a2f-4aa6-8501-4145a5e05517 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:46.491357 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e875036f-2a2f-4aa6-8501-4145a5e05517 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:46.491635 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e875036f-2a2f-4aa6-8501-4145a5e05517 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:46.510167 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-e875036f-2a2f-4aa6-8501-4145a5e05517 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:04:46.511363 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e875036f-2a2f-4aa6-8501-4145a5e05517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:46.521123 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 990/2006] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:04:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:04:46.528951 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed68c71-4595-4838-b9b9-b6798e476749 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:46.538536 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed68c71-4595-4838-b9b9-b6798e476749 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:46.543838 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b303c7-b320-4c3e-911d-d438f34d6cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:46.544520 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b303c7-b320-4c3e-911d-d438f34d6cfe admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:46.544803 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b303c7-b320-4c3e-911d-d438f34d6cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:46.545082 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b303c7-b320-4c3e-911d-d438f34d6cfe admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:46.545808 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b303c7-b320-4c3e-911d-d438f34d6cfe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 13:04:46.546116 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b303c7-b320-4c3e-911d-d438f34d6cfe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:46.547697 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b303c7-b320-4c3e-911d-d438f34d6cfe admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:46.563130 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1017/2007] 10.222.0.27 () {58 vars in 1152 bytes} [Fri Jun 10 13:04:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 13:04:46.564356 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e875036f-2a2f-4aa6-8501-4145a5e05517 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:46.573697 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e875036f-2a2f-4aa6-8501-4145a5e05517 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:46.578266 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c789062-c68f-4da5-a80d-ebc4000fc17e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:46.578919 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c789062-c68f-4da5-a80d-ebc4000fc17e admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:46.579205 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c789062-c68f-4da5-a80d-ebc4000fc17e admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:46.579480 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c789062-c68f-4da5-a80d-ebc4000fc17e admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:46.580201 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c789062-c68f-4da5-a80d-ebc4000fc17e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:46.582088 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c789062-c68f-4da5-a80d-ebc4000fc17e admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:46.679912 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 991/2008] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:04:46 2022] POST /identity/v3/projects => generated 383 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:04:46.682528 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b303c7-b320-4c3e-911d-d438f34d6cfe admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:46.692320 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b303c7-b320-4c3e-911d-d438f34d6cfe admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:46.748719 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b6e6bf-a88a-419c-b20f-c5b1619902b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:46.749318 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b6e6bf-a88a-419c-b20f-c5b1619902b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:46.749596 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b6e6bf-a88a-419c-b20f-c5b1619902b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:46.749867 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b6e6bf-a88a-419c-b20f-c5b1619902b7 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:46.750547 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b6e6bf-a88a-419c-b20f-c5b1619902b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:46.752527 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b6e6bf-a88a-419c-b20f-c5b1619902b7 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:46.760334 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c789062-c68f-4da5-a80d-ebc4000fc17e admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:46.763449 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-8c789062-c68f-4da5-a80d-ebc4000fc17e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:46.776537 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c789062-c68f-4da5-a80d-ebc4000fc17e admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:46.781096 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15636b2c-26c1-4f91-90f0-0fdcd39836bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:46.781960 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15636b2c-26c1-4f91-90f0-0fdcd39836bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:46.782264 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15636b2c-26c1-4f91-90f0-0fdcd39836bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:46.782545 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15636b2c-26c1-4f91-90f0-0fdcd39836bc admin admin] PATH_INFO: `/v3/users/116b4bdc30ae4054947ace2a82d01b01` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:46.783745 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15636b2c-26c1-4f91-90f0-0fdcd39836bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=116b4bdc30ae4054947ace2a82d01b01)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:46.785672 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15636b2c-26c1-4f91-90f0-0fdcd39836bc admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:46.877182 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1018/2009] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:04:46 2022] POST /identity/v3/users => generated 292 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:04:46.881433 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b6e6bf-a88a-419c-b20f-c5b1619902b7 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:46.891078 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b6e6bf-a88a-419c-b20f-c5b1619902b7 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:46.895853 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf3663e-52a7-489f-9578-05969911e495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:46.896539 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf3663e-52a7-489f-9578-05969911e495 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:46.896829 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf3663e-52a7-489f-9578-05969911e495 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:46.897109 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf3663e-52a7-489f-9578-05969911e495 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:46.898099 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf3663e-52a7-489f-9578-05969911e495 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:46.899767 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf3663e-52a7-489f-9578-05969911e495 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:46.910789 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1019/2010] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:04:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 10 13:04:46.915435 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf3663e-52a7-489f-9578-05969911e495 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:46.925784 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf3663e-52a7-489f-9578-05969911e495 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:46.930396 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ab2746-4061-4e5b-9b8e-0d457a1c7f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:46.931089 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ab2746-4061-4e5b-9b8e-0d457a1c7f0a admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:46.931378 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ab2746-4061-4e5b-9b8e-0d457a1c7f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:46.931658 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ab2746-4061-4e5b-9b8e-0d457a1c7f0a admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:46.941590 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ab2746-4061-4e5b-9b8e-0d457a1c7f0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:46.941590 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ab2746-4061-4e5b-9b8e-0d457a1c7f0a admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:46.948086 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1020/2011] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:04:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 10 13:04:46.949130 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ab2746-4061-4e5b-9b8e-0d457a1c7f0a admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:46.949254 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-15636b2c-26c1-4f91-90f0-0fdcd39836bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '116b4bdc30ae4054947ace2a82d01b01'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:46.955435 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:04:46.956123 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-15636b2c-26c1-4f91-90f0-0fdcd39836bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '116b4bdc30ae4054947ace2a82d01b01'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:46.959082 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ab2746-4061-4e5b-9b8e-0d457a1c7f0a admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:46.964234 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f4f2e2-780e-494b-a569-b45791b5d720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:46.965103 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f4f2e2-780e-494b-a569-b45791b5d720 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:46.965388 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f4f2e2-780e-494b-a569-b45791b5d720 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:46.965702 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f4f2e2-780e-494b-a569-b45791b5d720 admin admin] PATH_INFO: `/v3/projects/3fac54a887fe4da383319fa2c5f65235/users/16eac99a61d34b48b2d4232c1113e780/roles/43106c664e1b40de984c0c62701149c8` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:46.980398 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f4f2e2-780e-494b-a569-b45791b5d720 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fac54a887fe4da383319fa2c5f65235, user_id=16eac99a61d34b48b2d4232c1113e780, role_id=43106c664e1b40de984c0c62701149c8)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:46.983034 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f4f2e2-780e-494b-a569-b45791b5d720 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:47.047039 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-15636b2c-26c1-4f91-90f0-0fdcd39836bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '116b4bdc30ae4054947ace2a82d01b01'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:47.047605 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-15636b2c-26c1-4f91-90f0-0fdcd39836bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '116b4bdc30ae4054947ace2a82d01b01'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:47.052827 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1021/2012] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:04:46 2022] PUT /identity/v3/projects/3fac54a887fe4da383319fa2c5f65235/users/16eac99a61d34b48b2d4232c1113e780/roles/43106c664e1b40de984c0c62701149c8 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 10 13:04:47.060819 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-15636b2c-26c1-4f91-90f0-0fdcd39836bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '116b4bdc30ae4054947ace2a82d01b01'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:47.064386 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfec0f33-e24b-4c07-a953-08855317a0ad None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:47.064687 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfec0f33-e24b-4c07-a953-08855317a0ad None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:47.064987 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfec0f33-e24b-4c07-a953-08855317a0ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:47.067391 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 992/2013] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:04:46 2022] DELETE /identity/v3/users/116b4bdc30ae4054947ace2a82d01b01 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:04:47.071624 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867969c9-50d8-4116-8a6c-0cd312485ac3 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:47.071954 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867969c9-50d8-4116-8a6c-0cd312485ac3 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:47.072223 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867969c9-50d8-4116-8a6c-0cd312485ac3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:47.090661 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-867969c9-50d8-4116-8a6c-0cd312485ac3 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:04:47.091927 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-867969c9-50d8-4116-8a6c-0cd312485ac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:47.092913 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-dfec0f33-e24b-4c07-a953-08855317a0ad None None] MFA Rules not processed for user `16eac99a61d34b48b2d4232c1113e780`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:04:47.094176 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-dfec0f33-e24b-4c07-a953-08855317a0ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:47.155563 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 993/2014] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:04:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 10 13:04:47.160347 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1022/2015] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:04:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 10 13:04:47.164707 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867969c9-50d8-4116-8a6c-0cd312485ac3 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:47.165563 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfec0f33-e24b-4c07-a953-08855317a0ad None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:47.166045 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-867969c9-50d8-4116-8a6c-0cd312485ac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:47.167050 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-dfec0f33-e24b-4c07-a953-08855317a0ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:47.178136 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867969c9-50d8-4116-8a6c-0cd312485ac3 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:47.182945 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7261e374-cb02-4369-9745-c5b0017fe68d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:47.183598 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7261e374-cb02-4369-9745-c5b0017fe68d admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:47.183967 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7261e374-cb02-4369-9745-c5b0017fe68d admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:47.184187 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7261e374-cb02-4369-9745-c5b0017fe68d admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:47.184884 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7261e374-cb02-4369-9745-c5b0017fe68d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:47.186775 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7261e374-cb02-4369-9745-c5b0017fe68d admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:47.200581 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:04:47.225527 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfec0f33-e24b-4c07-a953-08855317a0ad None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:47.229435 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a57aaa-8cd0-40d3-a86d-5f370e27baff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:47.230077 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a57aaa-8cd0-40d3-a86d-5f370e27baff service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:47.230358 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a57aaa-8cd0-40d3-a86d-5f370e27baff service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:47.230632 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a57aaa-8cd0-40d3-a86d-5f370e27baff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:47.240102 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a57aaa-8cd0-40d3-a86d-5f370e27baff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:47.244184 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a57aaa-8cd0-40d3-a86d-5f370e27baff service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:47.250907 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1023/2016] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:04:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 10 13:04:47.273178 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 994/2017] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:04:47 2022] POST /identity/v3/projects => generated 383 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:04:47.275245 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a57aaa-8cd0-40d3-a86d-5f370e27baff service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:47.285358 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a57aaa-8cd0-40d3-a86d-5f370e27baff service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:47.351821 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06af870-0c51-4ff3-810b-a4d0617546a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:47.352537 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06af870-0c51-4ff3-810b-a4d0617546a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:47.352817 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06af870-0c51-4ff3-810b-a4d0617546a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:47.353096 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06af870-0c51-4ff3-810b-a4d0617546a6 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:47.353874 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06af870-0c51-4ff3-810b-a4d0617546a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:47.355825 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06af870-0c51-4ff3-810b-a4d0617546a6 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:47.468668 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1024/2018] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:04:47 2022] POST /identity/v3/users => generated 298 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:04:47.469992 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7261e374-cb02-4369-9745-c5b0017fe68d admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:47.481489 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7261e374-cb02-4369-9745-c5b0017fe68d admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:47.486366 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b694b6-d768-4f52-bb73-61808f2b79b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:47.486802 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b694b6-d768-4f52-bb73-61808f2b79b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:47.487120 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b694b6-d768-4f52-bb73-61808f2b79b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:47.487391 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b694b6-d768-4f52-bb73-61808f2b79b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:47.488303 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b694b6-d768-4f52-bb73-61808f2b79b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:47.489896 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b694b6-d768-4f52-bb73-61808f2b79b1 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:47.502741 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 995/2019] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:04:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:04:47.506831 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06af870-0c51-4ff3-810b-a4d0617546a6 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:47.513072 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:04:47.518119 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06af870-0c51-4ff3-810b-a4d0617546a6 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:47.523173 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fdb08c-6154-4a0e-980d-2eed7c3bfd11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:47.524156 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fdb08c-6154-4a0e-980d-2eed7c3bfd11 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:47.524436 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fdb08c-6154-4a0e-980d-2eed7c3bfd11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:47.524710 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fdb08c-6154-4a0e-980d-2eed7c3bfd11 admin admin] PATH_INFO: `/v3/projects/5726ef9ae4eb41419b669f122be6729f/users/4082d747bc014ea0a5674de7b2010b14/roles/b43d5a8247e244869c2c70f986f42e8a` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:47.539503 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fdb08c-6154-4a0e-980d-2eed7c3bfd11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5726ef9ae4eb41419b669f122be6729f, user_id=4082d747bc014ea0a5674de7b2010b14, role_id=b43d5a8247e244869c2c70f986f42e8a)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:47.542740 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fdb08c-6154-4a0e-980d-2eed7c3bfd11 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:47.614828 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1025/2020] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:04:47 2022] PUT /identity/v3/projects/5726ef9ae4eb41419b669f122be6729f/users/4082d747bc014ea0a5674de7b2010b14/roles/b43d5a8247e244869c2c70f986f42e8a => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:04:47.618696 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b694b6-d768-4f52-bb73-61808f2b79b1 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:47.630879 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b694b6-d768-4f52-bb73-61808f2b79b1 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:47.690148 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fdb08c-6154-4a0e-980d-2eed7c3bfd11 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:47.691922 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-15fdb08c-6154-4a0e-980d-2eed7c3bfd11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:47.695562 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21d0c0b-a5e3-4c01-911c-96e31e336fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:47.702415 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21d0c0b-a5e3-4c01-911c-96e31e336fc4 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:47.702415 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21d0c0b-a5e3-4c01-911c-96e31e336fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:47.702415 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21d0c0b-a5e3-4c01-911c-96e31e336fc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:47.702415 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21d0c0b-a5e3-4c01-911c-96e31e336fc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:47.702415 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21d0c0b-a5e3-4c01-911c-96e31e336fc4 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:47.705663 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fdb08c-6154-4a0e-980d-2eed7c3bfd11 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:47.710573 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f41286a-1cdc-4c2d-beeb-877fe45fe331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:47.711443 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f41286a-1cdc-4c2d-beeb-877fe45fe331 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:47.711751 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f41286a-1cdc-4c2d-beeb-877fe45fe331 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:47.712052 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f41286a-1cdc-4c2d-beeb-877fe45fe331 admin admin] PATH_INFO: `/v3/projects/3dfbc1605a404a10a4f6428ee51032e4` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:47.713497 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f41286a-1cdc-4c2d-beeb-877fe45fe331 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3dfbc1605a404a10a4f6428ee51032e4)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:47.713720 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 996/2021] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:04:47 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:04:47.717650 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21d0c0b-a5e3-4c01-911c-96e31e336fc4 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:47.718292 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f41286a-1cdc-4c2d-beeb-877fe45fe331 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:47.729551 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21d0c0b-a5e3-4c01-911c-96e31e336fc4 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:47.734135 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750abcf9-8067-45cf-8b99-c567cd8927b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:47.734998 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750abcf9-8067-45cf-8b99-c567cd8927b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:47.735395 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750abcf9-8067-45cf-8b99-c567cd8927b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:47.735549 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750abcf9-8067-45cf-8b99-c567cd8927b8 admin admin] PATH_INFO: `/v3/projects/5726ef9ae4eb41419b669f122be6729f/users/4082d747bc014ea0a5674de7b2010b14/roles/b43d5a8247e244869c2c70f986f42e8a` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:47.737650 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750abcf9-8067-45cf-8b99-c567cd8927b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5726ef9ae4eb41419b669f122be6729f, user_id=4082d747bc014ea0a5674de7b2010b14, role_id=b43d5a8247e244869c2c70f986f42e8a)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:47.740841 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750abcf9-8067-45cf-8b99-c567cd8927b8 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:47.751901 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 997/2022] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:04:47 2022] PUT /identity/v3/projects/5726ef9ae4eb41419b669f122be6729f/users/4082d747bc014ea0a5674de7b2010b14/roles/b43d5a8247e244869c2c70f986f42e8a => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 10 13:04:47.762166 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3311ae71-e259-4554-a265-0c4387d749c8 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:47.762506 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3311ae71-e259-4554-a265-0c4387d749c8 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:47.762787 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3311ae71-e259-4554-a265-0c4387d749c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:47.788643 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-3311ae71-e259-4554-a265-0c4387d749c8 None None] MFA Rules not processed for user `4082d747bc014ea0a5674de7b2010b14`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:04:47.789869 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-3311ae71-e259-4554-a265-0c4387d749c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:47.836887 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5f41286a-1cdc-4c2d-beeb-877fe45fe331 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3dfbc1605a404a10a4f6428ee51032e4', 'request_id': 'req-5f41286a-1cdc-4c2d-beeb-877fe45fe331'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:47.842662 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5f41286a-1cdc-4c2d-beeb-877fe45fe331 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3dfbc1605a404a10a4f6428ee51032e4', 'request_id': 'req-5f41286a-1cdc-4c2d-beeb-877fe45fe331'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:47.849494 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5f41286a-1cdc-4c2d-beeb-877fe45fe331 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3dfbc1605a404a10a4f6428ee51032e4', 'request_id': 'req-5f41286a-1cdc-4c2d-beeb-877fe45fe331'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:47.858349 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 998/2023] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:04:47 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 10 13:04:47.868890 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3311ae71-e259-4554-a265-0c4387d749c8 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:48.209039 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-3311ae71-e259-4554-a265-0c4387d749c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:48.209039 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3311ae71-e259-4554-a265-0c4387d749c8 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:48.209039 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:04:48.209039 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d0ad2b-c80f-4876-9e2c-03a479aa3afb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6cc95a563b941ac92ab283aa0d5a5d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:48.210499 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d0ad2b-c80f-4876-9e2c-03a479aa3afb service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:48.210499 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d0ad2b-c80f-4876-9e2c-03a479aa3afb service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:48.210499 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d0ad2b-c80f-4876-9e2c-03a479aa3afb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:48.211209 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5f41286a-1cdc-4c2d-beeb-877fe45fe331 admin admin] The token cache is being invalidate because project 3dfbc1605a404a10a4f6428ee51032e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:04:48.211209 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5f41286a-1cdc-4c2d-beeb-877fe45fe331 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:48.211209 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5f41286a-1cdc-4c2d-beeb-877fe45fe331 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:48.211209 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1026/2024] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:04:47 2022] DELETE /identity/v3/projects/3dfbc1605a404a10a4f6428ee51032e4 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 10 13:04:48.211209 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f41286a-1cdc-4c2d-beeb-877fe45fe331 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:48.212125 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d0ad2b-c80f-4876-9e2c-03a479aa3afb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:48.212320 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5f41286a-1cdc-4c2d-beeb-877fe45fe331 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:48.212511 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d0ad2b-c80f-4876-9e2c-03a479aa3afb service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:48.212511 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 999/2025] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:04:47 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 10 13:04:48.214373 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f41286a-1cdc-4c2d-beeb-877fe45fe331 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:48.214373 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c6a364-f230-4668-bf1d-d43d4f823f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:48.214373 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c6a364-f230-4668-bf1d-d43d4f823f16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:48.214373 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c6a364-f230-4668-bf1d-d43d4f823f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:48.214708 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c6a364-f230-4668-bf1d-d43d4f823f16 admin admin] PATH_INFO: `/v3/users/196fcd8baa3e4468b2a321119c1224f0` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:48.214708 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c6a364-f230-4668-bf1d-d43d4f823f16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=196fcd8baa3e4468b2a321119c1224f0)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:48.214708 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c6a364-f230-4668-bf1d-d43d4f823f16 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:48.214950 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d0ad2b-c80f-4876-9e2c-03a479aa3afb service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:48.214950 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-43d0ad2b-c80f-4876-9e2c-03a479aa3afb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:48.214950 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d0ad2b-c80f-4876-9e2c-03a479aa3afb service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:48.215221 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f40dfe-cdc2-40a1-9028-d9726dddada8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6cc95a563b941ac92ab283aa0d5a5d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:48.215221 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f40dfe-cdc2-40a1-9028-d9726dddada8 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:48.215221 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f40dfe-cdc2-40a1-9028-d9726dddada8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:48.215221 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f40dfe-cdc2-40a1-9028-d9726dddada8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:48.215550 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-12f40dfe-cdc2-40a1-9028-d9726dddada8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:48.232111 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f40dfe-cdc2-40a1-9028-d9726dddada8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:48.235014 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f40dfe-cdc2-40a1-9028-d9726dddada8 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:48.242688 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1000/2026] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:04:48 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 10 13:04:48.371485 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-52c6a364-f230-4668-bf1d-d43d4f823f16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '196fcd8baa3e4468b2a321119c1224f0'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:48.378175 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-52c6a364-f230-4668-bf1d-d43d4f823f16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '196fcd8baa3e4468b2a321119c1224f0'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:48.459553 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-52c6a364-f230-4668-bf1d-d43d4f823f16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '196fcd8baa3e4468b2a321119c1224f0'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:48.460171 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-52c6a364-f230-4668-bf1d-d43d4f823f16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '196fcd8baa3e4468b2a321119c1224f0'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:48.461638 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=82,keystone:INSERT=3,keystone:DELETE=15 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:04:48.483659 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-52c6a364-f230-4668-bf1d-d43d4f823f16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '196fcd8baa3e4468b2a321119c1224f0'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:48.496333 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1027/2027] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:04:48 2022] DELETE /identity/v3/users/196fcd8baa3e4468b2a321119c1224f0 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 10 13:04:48.918046 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f40dfe-cdc2-40a1-9028-d9726dddada8 service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:48.930477 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f40dfe-cdc2-40a1-9028-d9726dddada8 service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:48.985765 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:04:49.000558 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2027d1e7-3da8-43ea-8832-aa42b8351ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:49.006262 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2027d1e7-3da8-43ea-8832-aa42b8351ed1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:49.006262 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2027d1e7-3da8-43ea-8832-aa42b8351ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:49.006262 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2027d1e7-3da8-43ea-8832-aa42b8351ed1 admin admin] PATH_INFO: `/v3/projects/bcee24bbd42540c692f0fb14e8d39b79` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:49.006262 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2027d1e7-3da8-43ea-8832-aa42b8351ed1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bcee24bbd42540c692f0fb14e8d39b79)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:49.006262 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2027d1e7-3da8-43ea-8832-aa42b8351ed1 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:49.133782 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2027d1e7-3da8-43ea-8832-aa42b8351ed1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bcee24bbd42540c692f0fb14e8d39b79', 'request_id': 'req-2027d1e7-3da8-43ea-8832-aa42b8351ed1'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:49.139755 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2027d1e7-3da8-43ea-8832-aa42b8351ed1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bcee24bbd42540c692f0fb14e8d39b79', 'request_id': 'req-2027d1e7-3da8-43ea-8832-aa42b8351ed1'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:49.148397 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2027d1e7-3da8-43ea-8832-aa42b8351ed1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bcee24bbd42540c692f0fb14e8d39b79', 'request_id': 'req-2027d1e7-3da8-43ea-8832-aa42b8351ed1'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:49.248510 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2027d1e7-3da8-43ea-8832-aa42b8351ed1 admin admin] The token cache is being invalidate because project bcee24bbd42540c692f0fb14e8d39b79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:04:49.249133 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2027d1e7-3da8-43ea-8832-aa42b8351ed1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:49.249332 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2027d1e7-3da8-43ea-8832-aa42b8351ed1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:49.250896 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1001/2028] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:04:48 2022] DELETE /identity/v3/projects/bcee24bbd42540c692f0fb14e8d39b79 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:04:49.256667 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5540e98-61c4-4c13-9785-b82b7552f754 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:49.256814 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5540e98-61c4-4c13-9785-b82b7552f754 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:49.257011 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5540e98-61c4-4c13-9785-b82b7552f754 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:49.277135 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-d5540e98-61c4-4c13-9785-b82b7552f754 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:04:49.286305 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d5540e98-61c4-4c13-9785-b82b7552f754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:49.339449 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1028/2029] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:04:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:04:49.347184 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf0c4d5-7151-4a3b-818d-5c0fcfe0954f None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:49.347482 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf0c4d5-7151-4a3b-818d-5c0fcfe0954f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:49.347772 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf0c4d5-7151-4a3b-818d-5c0fcfe0954f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:49.366601 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-4cf0c4d5-7151-4a3b-818d-5c0fcfe0954f None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:04:49.367775 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-4cf0c4d5-7151-4a3b-818d-5c0fcfe0954f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:49.376438 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1002/2030] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:04:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:04:49.380646 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5540e98-61c4-4c13-9785-b82b7552f754 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:49.390998 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5540e98-61c4-4c13-9785-b82b7552f754 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:49.395676 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05572f01-ec7e-4ca4-afb0-b6c8926b8d6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:49.396372 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05572f01-ec7e-4ca4-afb0-b6c8926b8d6e admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:49.396655 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05572f01-ec7e-4ca4-afb0-b6c8926b8d6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:49.396949 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05572f01-ec7e-4ca4-afb0-b6c8926b8d6e admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:49.397952 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05572f01-ec7e-4ca4-afb0-b6c8926b8d6e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 13:04:49.398254 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05572f01-ec7e-4ca4-afb0-b6c8926b8d6e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:49.399834 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05572f01-ec7e-4ca4-afb0-b6c8926b8d6e admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:49.412401 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1029/2031] 10.222.0.27 () {58 vars in 1152 bytes} [Fri Jun 10 13:04:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 13:04:49.417263 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf0c4d5-7151-4a3b-818d-5c0fcfe0954f None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:49.426484 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf0c4d5-7151-4a3b-818d-5c0fcfe0954f None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:49.431154 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee160cf-1d1f-45b1-9e60-8fefb350ecbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:49.431806 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee160cf-1d1f-45b1-9e60-8fefb350ecbd admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:49.432792 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee160cf-1d1f-45b1-9e60-8fefb350ecbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:49.433072 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee160cf-1d1f-45b1-9e60-8fefb350ecbd admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:49.433773 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee160cf-1d1f-45b1-9e60-8fefb350ecbd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:49.435671 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee160cf-1d1f-45b1-9e60-8fefb350ecbd admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:49.539730 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05572f01-ec7e-4ca4-afb0-b6c8926b8d6e admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:49.540283 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1003/2032] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:04:49 2022] POST /identity/v3/projects => generated 387 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:04:49.544076 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee160cf-1d1f-45b1-9e60-8fefb350ecbd admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:49.544541 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-05572f01-ec7e-4ca4-afb0-b6c8926b8d6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:49.558431 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05572f01-ec7e-4ca4-afb0-b6c8926b8d6e admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:49.605262 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee160cf-1d1f-45b1-9e60-8fefb350ecbd admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:49.610178 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2587d041-8bee-4a74-861d-9d11e30eb4c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:49.610854 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2587d041-8bee-4a74-861d-9d11e30eb4c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:49.611140 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2587d041-8bee-4a74-861d-9d11e30eb4c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:49.611419 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2587d041-8bee-4a74-861d-9d11e30eb4c7 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:49.612217 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2587d041-8bee-4a74-861d-9d11e30eb4c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:49.616407 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2587d041-8bee-4a74-861d-9d11e30eb4c7 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:49.616511 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7608a9f5-564a-4593-9a94-3b01d5dbdf1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:49.616511 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7608a9f5-564a-4593-9a94-3b01d5dbdf1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:49.616714 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7608a9f5-564a-4593-9a94-3b01d5dbdf1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:49.616971 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7608a9f5-564a-4593-9a94-3b01d5dbdf1f admin admin] PATH_INFO: `/v3/users/5b58c9a7fcea4e75920923e3caba48b9` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:49.618193 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7608a9f5-564a-4593-9a94-3b01d5dbdf1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b58c9a7fcea4e75920923e3caba48b9)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:49.620802 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7608a9f5-564a-4593-9a94-3b01d5dbdf1f admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:49.723266 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1004/2033] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:04:49 2022] POST /identity/v3/users => generated 294 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 10 13:04:49.727079 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2587d041-8bee-4a74-861d-9d11e30eb4c7 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:49.738402 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2587d041-8bee-4a74-861d-9d11e30eb4c7 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:49.743335 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86719fbf-98e4-4af7-bfb2-68c8765322f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:49.747210 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86719fbf-98e4-4af7-bfb2-68c8765322f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:49.747210 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86719fbf-98e4-4af7-bfb2-68c8765322f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:49.747210 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86719fbf-98e4-4af7-bfb2-68c8765322f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:49.747210 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86719fbf-98e4-4af7-bfb2-68c8765322f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:49.747659 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86719fbf-98e4-4af7-bfb2-68c8765322f6 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:49.762787 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1005/2034] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:04:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 10 13:04:49.763784 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86719fbf-98e4-4af7-bfb2-68c8765322f6 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:49.779136 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86719fbf-98e4-4af7-bfb2-68c8765322f6 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:49.779136 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b9a26e-da8c-4ef5-913b-fe90cef3ba9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:49.779625 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b9a26e-da8c-4ef5-913b-fe90cef3ba9c admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:49.779872 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b9a26e-da8c-4ef5-913b-fe90cef3ba9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:49.780161 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b9a26e-da8c-4ef5-913b-fe90cef3ba9c admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:49.780882 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b9a26e-da8c-4ef5-913b-fe90cef3ba9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:49.782455 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b9a26e-da8c-4ef5-913b-fe90cef3ba9c admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:49.793593 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1006/2035] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:04:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:04:49.798676 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b9a26e-da8c-4ef5-913b-fe90cef3ba9c admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:49.808154 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b9a26e-da8c-4ef5-913b-fe90cef3ba9c admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:49.812882 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e51f840-1498-44d2-a1ce-a52fc6071d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:49.813696 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e51f840-1498-44d2-a1ce-a52fc6071d2b admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:49.813977 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e51f840-1498-44d2-a1ce-a52fc6071d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:49.814246 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e51f840-1498-44d2-a1ce-a52fc6071d2b admin admin] PATH_INFO: `/v3/projects/264db64881c14d389b1c8f5cdc6cf1fb/users/7a06196d87094308b80547966826614d/roles/43106c664e1b40de984c0c62701149c8` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:49.821452 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-7608a9f5-564a-4593-9a94-3b01d5dbdf1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b58c9a7fcea4e75920923e3caba48b9'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:49.828575 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-7608a9f5-564a-4593-9a94-3b01d5dbdf1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b58c9a7fcea4e75920923e3caba48b9'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:49.828814 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e51f840-1498-44d2-a1ce-a52fc6071d2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=264db64881c14d389b1c8f5cdc6cf1fb, user_id=7a06196d87094308b80547966826614d, role_id=43106c664e1b40de984c0c62701149c8)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:49.831463 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e51f840-1498-44d2-a1ce-a52fc6071d2b admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:49.887969 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-7608a9f5-564a-4593-9a94-3b01d5dbdf1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b58c9a7fcea4e75920923e3caba48b9'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:49.888611 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-7608a9f5-564a-4593-9a94-3b01d5dbdf1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b58c9a7fcea4e75920923e3caba48b9'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:49.894323 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:04:49.901407 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=87,keystone:DELETE=7,keystone:INSERT=6 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:04:49.902494 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-7608a9f5-564a-4593-9a94-3b01d5dbdf1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b58c9a7fcea4e75920923e3caba48b9'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:49.906263 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1007/2036] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:04:49 2022] PUT /identity/v3/projects/264db64881c14d389b1c8f5cdc6cf1fb/users/7a06196d87094308b80547966826614d/roles/43106c664e1b40de984c0c62701149c8 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 10 13:04:49.915717 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1030/2037] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:04:49 2022] DELETE /identity/v3/users/5b58c9a7fcea4e75920923e3caba48b9 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 10 13:04:49.919094 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc69f8d-8345-4d23-886f-e585897f213e None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:49.919094 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc69f8d-8345-4d23-886f-e585897f213e None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:49.919094 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc69f8d-8345-4d23-886f-e585897f213e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:49.920615 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d112524-c412-4a23-b63f-6933a66e0e5a None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:49.920901 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d112524-c412-4a23-b63f-6933a66e0e5a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:49.921178 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d112524-c412-4a23-b63f-6933a66e0e5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:49.938637 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-3d112524-c412-4a23-b63f-6933a66e0e5a None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:04:49.939835 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3d112524-c412-4a23-b63f-6933a66e0e5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:49.945388 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-3bc69f8d-8345-4d23-886f-e585897f213e None None] MFA Rules not processed for user `7a06196d87094308b80547966826614d`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:04:49.946614 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-3bc69f8d-8345-4d23-886f-e585897f213e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:49.999946 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1031/2038] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:04:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 10 13:04:50.012774 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d112524-c412-4a23-b63f-6933a66e0e5a None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:50.013274 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1008/2039] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:04:49 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 10 13:04:50.015185 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3d112524-c412-4a23-b63f-6933a66e0e5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:50.017737 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc69f8d-8345-4d23-886f-e585897f213e None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:50.018983 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-3bc69f8d-8345-4d23-886f-e585897f213e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:50.033196 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc69f8d-8345-4d23-886f-e585897f213e None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:50.039047 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7eb096-ec92-4526-add7-ff43fb098b0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:50.039704 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7eb096-ec92-4526-add7-ff43fb098b0e admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:50.040018 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7eb096-ec92-4526-add7-ff43fb098b0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:50.040291 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7eb096-ec92-4526-add7-ff43fb098b0e admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:50.041051 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7eb096-ec92-4526-add7-ff43fb098b0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:50.043265 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7eb096-ec92-4526-add7-ff43fb098b0e admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:50.076958 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d112524-c412-4a23-b63f-6933a66e0e5a None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:50.080734 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68398600-0fee-4d15-b1c3-ecbee30efb98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:50.081372 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68398600-0fee-4d15-b1c3-ecbee30efb98 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:50.081654 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68398600-0fee-4d15-b1c3-ecbee30efb98 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:50.081928 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68398600-0fee-4d15-b1c3-ecbee30efb98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:50.091434 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68398600-0fee-4d15-b1c3-ecbee30efb98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:50.094057 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68398600-0fee-4d15-b1c3-ecbee30efb98 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:50.102038 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1032/2040] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:04:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:04:50.123896 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1009/2041] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:04:50 2022] POST /identity/v3/projects => generated 387 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 10 13:04:50.126030 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68398600-0fee-4d15-b1c3-ecbee30efb98 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:50.136073 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68398600-0fee-4d15-b1c3-ecbee30efb98 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:50.195171 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad60305-1cbc-4712-9334-257a48da9599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:50.195847 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad60305-1cbc-4712-9334-257a48da9599 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:50.196165 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad60305-1cbc-4712-9334-257a48da9599 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:50.196472 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad60305-1cbc-4712-9334-257a48da9599 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:50.197168 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad60305-1cbc-4712-9334-257a48da9599 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:50.199574 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad60305-1cbc-4712-9334-257a48da9599 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:50.305776 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1033/2042] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:04:50 2022] POST /identity/v3/users => generated 300 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 10 13:04:50.309808 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7eb096-ec92-4526-add7-ff43fb098b0e admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:50.323170 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7eb096-ec92-4526-add7-ff43fb098b0e admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:50.328717 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2f7032-2634-40ae-a3b7-90fea6566964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:50.329506 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2f7032-2634-40ae-a3b7-90fea6566964 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:50.329794 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2f7032-2634-40ae-a3b7-90fea6566964 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:50.330071 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2f7032-2634-40ae-a3b7-90fea6566964 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:50.330801 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2f7032-2634-40ae-a3b7-90fea6566964 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:50.332472 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2f7032-2634-40ae-a3b7-90fea6566964 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:50.347970 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1010/2043] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:04:50 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 10 13:04:50.349260 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad60305-1cbc-4712-9334-257a48da9599 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:50.360053 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad60305-1cbc-4712-9334-257a48da9599 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:50.364848 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809e56c4-4740-4d97-b6c6-fb2813c448ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:50.365925 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809e56c4-4740-4d97-b6c6-fb2813c448ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:50.366213 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809e56c4-4740-4d97-b6c6-fb2813c448ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:50.366495 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809e56c4-4740-4d97-b6c6-fb2813c448ea admin admin] PATH_INFO: `/v3/projects/545b53cb243d4428a207594c827f9931/users/478599ffb1684a2abdc89875a56b9b84/roles/b43d5a8247e244869c2c70f986f42e8a` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:50.384020 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809e56c4-4740-4d97-b6c6-fb2813c448ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=545b53cb243d4428a207594c827f9931, user_id=478599ffb1684a2abdc89875a56b9b84, role_id=b43d5a8247e244869c2c70f986f42e8a)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:50.397524 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809e56c4-4740-4d97-b6c6-fb2813c448ea admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:50.397524 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=95,keystone:DELETE=1,keystone:INSERT=4 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:04:50.546552 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1034/2044] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:04:50 2022] PUT /identity/v3/projects/545b53cb243d4428a207594c827f9931/users/478599ffb1684a2abdc89875a56b9b84/roles/b43d5a8247e244869c2c70f986f42e8a => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:04:50.550091 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2f7032-2634-40ae-a3b7-90fea6566964 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:50.550091 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-fb2f7032-2634-40ae-a3b7-90fea6566964 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:50.550332 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809e56c4-4740-4d97-b6c6-fb2813c448ea admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:50.579306 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2f7032-2634-40ae-a3b7-90fea6566964 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:50.614147 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809e56c4-4740-4d97-b6c6-fb2813c448ea admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:50.618897 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f346830-d98e-4aa9-983c-76601833f106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:50.620108 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f346830-d98e-4aa9-983c-76601833f106 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:50.620392 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f346830-d98e-4aa9-983c-76601833f106 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:50.620673 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f346830-d98e-4aa9-983c-76601833f106 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:50.621393 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f346830-d98e-4aa9-983c-76601833f106 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:50.623135 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f346830-d98e-4aa9-983c-76601833f106 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:50.632602 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edb8003-8923-4a47-b8a0-63386c03242f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:50.636969 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edb8003-8923-4a47-b8a0-63386c03242f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:50.637083 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1035/2045] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:04:50 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 10 13:04:50.637170 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edb8003-8923-4a47-b8a0-63386c03242f admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:50.637170 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edb8003-8923-4a47-b8a0-63386c03242f admin admin] PATH_INFO: `/v3/projects/b6d1773e054c476b941c5661bc1fad5b` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:50.637170 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edb8003-8923-4a47-b8a0-63386c03242f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6d1773e054c476b941c5661bc1fad5b)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:50.637758 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edb8003-8923-4a47-b8a0-63386c03242f admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:50.637961 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f346830-d98e-4aa9-983c-76601833f106 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:50.650886 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f346830-d98e-4aa9-983c-76601833f106 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:50.655517 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581054d2-be2e-4fb0-b7b8-aaa0bfbfed3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:50.656445 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581054d2-be2e-4fb0-b7b8-aaa0bfbfed3b admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:50.656725 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581054d2-be2e-4fb0-b7b8-aaa0bfbfed3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:50.656999 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581054d2-be2e-4fb0-b7b8-aaa0bfbfed3b admin admin] PATH_INFO: `/v3/projects/545b53cb243d4428a207594c827f9931/users/478599ffb1684a2abdc89875a56b9b84/roles/b43d5a8247e244869c2c70f986f42e8a` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:50.658996 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581054d2-be2e-4fb0-b7b8-aaa0bfbfed3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=545b53cb243d4428a207594c827f9931, user_id=478599ffb1684a2abdc89875a56b9b84, role_id=b43d5a8247e244869c2c70f986f42e8a)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:50.661625 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581054d2-be2e-4fb0-b7b8-aaa0bfbfed3b admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:50.671981 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1036/2046] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:04:50 2022] PUT /identity/v3/projects/545b53cb243d4428a207594c827f9931/users/478599ffb1684a2abdc89875a56b9b84/roles/b43d5a8247e244869c2c70f986f42e8a => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 10 13:04:50.681845 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f1f064-1b87-4ac6-999a-52ac21a5c782 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:50.682635 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f1f064-1b87-4ac6-999a-52ac21a5c782 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:50.682941 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f1f064-1b87-4ac6-999a-52ac21a5c782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:50.708829 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-78f1f064-1b87-4ac6-999a-52ac21a5c782 None None] MFA Rules not processed for user `478599ffb1684a2abdc89875a56b9b84`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:04:50.710034 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-78f1f064-1b87-4ac6-999a-52ac21a5c782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:50.751459 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-1edb8003-8923-4a47-b8a0-63386c03242f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6d1773e054c476b941c5661bc1fad5b', 'request_id': 'req-1edb8003-8923-4a47-b8a0-63386c03242f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:50.757401 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-1edb8003-8923-4a47-b8a0-63386c03242f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6d1773e054c476b941c5661bc1fad5b', 'request_id': 'req-1edb8003-8923-4a47-b8a0-63386c03242f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:50.758634 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:04:50.765765 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-1edb8003-8923-4a47-b8a0-63386c03242f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6d1773e054c476b941c5661bc1fad5b', 'request_id': 'req-1edb8003-8923-4a47-b8a0-63386c03242f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:50.781490 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1037/2047] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:04:50 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 10 13:04:50.792607 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f1f064-1b87-4ac6-999a-52ac21a5c782 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:50.793921 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-78f1f064-1b87-4ac6-999a-52ac21a5c782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:50.852368 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f1f064-1b87-4ac6-999a-52ac21a5c782 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:50.878153 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:04:50.878666 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-1edb8003-8923-4a47-b8a0-63386c03242f admin admin] The token cache is being invalidate because project b6d1773e054c476b941c5661bc1fad5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:04:50.879137 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-1edb8003-8923-4a47-b8a0-63386c03242f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:50.879846 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-1edb8003-8923-4a47-b8a0-63386c03242f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:50.882070 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1011/2048] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:04:50 2022] DELETE /identity/v3/projects/b6d1773e054c476b941c5661bc1fad5b => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:04:50.887128 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f43724d-5782-41ad-966e-18e244473d15 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:50.887440 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f43724d-5782-41ad-966e-18e244473d15 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:50.887697 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f43724d-5782-41ad-966e-18e244473d15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:50.910439 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-8f43724d-5782-41ad-966e-18e244473d15 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:04:50.911825 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-8f43724d-5782-41ad-966e-18e244473d15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:50.915382 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5c6f1a-49e2-424a-8b6a-bce6e8975618 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6cc95a563b941ac92ab283aa0d5a5d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:50.916842 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5c6f1a-49e2-424a-8b6a-bce6e8975618 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:50.917207 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5c6f1a-49e2-424a-8b6a-bce6e8975618 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:50.917484 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5c6f1a-49e2-424a-8b6a-bce6e8975618 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:50.919408 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3c5c6f1a-49e2-424a-8b6a-bce6e8975618 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:50.984228 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1012/2049] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:04:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 10 13:04:50.984429 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5c6f1a-49e2-424a-8b6a-bce6e8975618 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:50.985843 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5c6f1a-49e2-424a-8b6a-bce6e8975618 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:50.991397 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31555862-78c6-449f-9569-25f0902ebf9b None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:50.991692 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31555862-78c6-449f-9569-25f0902ebf9b None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:50.992032 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31555862-78c6-449f-9569-25f0902ebf9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:50.996237 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1038/2050] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:04:50 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 10 13:04:51.011080 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-31555862-78c6-449f-9569-25f0902ebf9b None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:04:51.012452 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-31555862-78c6-449f-9569-25f0902ebf9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:51.020872 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1013/2051] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:04:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 10 13:04:51.025330 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5c6f1a-49e2-424a-8b6a-bce6e8975618 service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:51.035546 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5c6f1a-49e2-424a-8b6a-bce6e8975618 service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:51.040163 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b21be6-bdb5-4a5c-b47d-da659e422edc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:51.040834 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b21be6-bdb5-4a5c-b47d-da659e422edc admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:51.041114 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b21be6-bdb5-4a5c-b47d-da659e422edc admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:51.041993 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b21be6-bdb5-4a5c-b47d-da659e422edc admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:51.042718 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b21be6-bdb5-4a5c-b47d-da659e422edc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 13:04:51.043022 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b21be6-bdb5-4a5c-b47d-da659e422edc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:51.044623 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b21be6-bdb5-4a5c-b47d-da659e422edc admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:51.055829 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1039/2052] 10.222.0.27 () {58 vars in 1152 bytes} [Fri Jun 10 13:04:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 13:04:51.064760 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31555862-78c6-449f-9569-25f0902ebf9b None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:51.070361 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31555862-78c6-449f-9569-25f0902ebf9b None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:51.075066 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a577519-0368-4b18-ac35-9a6b49221a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:51.075729 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a577519-0368-4b18-ac35-9a6b49221a41 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:51.076042 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a577519-0368-4b18-ac35-9a6b49221a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:51.076324 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a577519-0368-4b18-ac35-9a6b49221a41 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:51.077015 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a577519-0368-4b18-ac35-9a6b49221a41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:51.079004 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a577519-0368-4b18-ac35-9a6b49221a41 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:51.177506 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1014/2053] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:04:51 2022] POST /identity/v3/projects => generated 359 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 10 13:04:51.181763 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b21be6-bdb5-4a5c-b47d-da659e422edc admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:51.192483 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b21be6-bdb5-4a5c-b47d-da659e422edc admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:51.252254 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb65ca9-976a-41da-8347-2ca0e82035c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:51.252951 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb65ca9-976a-41da-8347-2ca0e82035c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:51.253315 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb65ca9-976a-41da-8347-2ca0e82035c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:51.253661 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb65ca9-976a-41da-8347-2ca0e82035c9 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:51.254465 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb65ca9-976a-41da-8347-2ca0e82035c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:51.256422 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb65ca9-976a-41da-8347-2ca0e82035c9 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:51.379008 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1040/2054] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:04:51 2022] POST /identity/v3/users => generated 280 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:04:51.383076 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a577519-0368-4b18-ac35-9a6b49221a41 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:51.393347 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a577519-0368-4b18-ac35-9a6b49221a41 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:51.398465 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6641d7d2-e6ff-41e3-8489-1a5f396c6b0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:51.399131 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6641d7d2-e6ff-41e3-8489-1a5f396c6b0c admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:51.399442 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6641d7d2-e6ff-41e3-8489-1a5f396c6b0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:51.399716 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6641d7d2-e6ff-41e3-8489-1a5f396c6b0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:51.400527 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6641d7d2-e6ff-41e3-8489-1a5f396c6b0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:51.405661 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6641d7d2-e6ff-41e3-8489-1a5f396c6b0c admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:51.413172 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1015/2055] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:04:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:04:51.417570 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb65ca9-976a-41da-8347-2ca0e82035c9 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:51.427619 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb65ca9-976a-41da-8347-2ca0e82035c9 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:51.434506 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fdcd36-b68a-4ab3-914c-86a6fbdced9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:51.434636 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fdcd36-b68a-4ab3-914c-86a6fbdced9e admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:51.434867 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fdcd36-b68a-4ab3-914c-86a6fbdced9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:51.435143 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fdcd36-b68a-4ab3-914c-86a6fbdced9e admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:51.435888 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fdcd36-b68a-4ab3-914c-86a6fbdced9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:51.437496 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fdcd36-b68a-4ab3-914c-86a6fbdced9e admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:51.450750 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1041/2056] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:04:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:04:51.452064 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6641d7d2-e6ff-41e3-8489-1a5f396c6b0c admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:51.465738 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6641d7d2-e6ff-41e3-8489-1a5f396c6b0c admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:51.470791 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfa4ae8-51e7-4c30-ac60-54dd9e3b9d5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:51.471681 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfa4ae8-51e7-4c30-ac60-54dd9e3b9d5a admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:51.471995 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfa4ae8-51e7-4c30-ac60-54dd9e3b9d5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:51.472361 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfa4ae8-51e7-4c30-ac60-54dd9e3b9d5a admin admin] PATH_INFO: `/v3/projects/04686fb6d1184bc39b630033fc63d83a/users/ef37d962f6a245da841eaf1959ec0aa7/roles/43106c664e1b40de984c0c62701149c8` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:51.489321 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfa4ae8-51e7-4c30-ac60-54dd9e3b9d5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=04686fb6d1184bc39b630033fc63d83a, user_id=ef37d962f6a245da841eaf1959ec0aa7, role_id=43106c664e1b40de984c0c62701149c8)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:51.492030 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfa4ae8-51e7-4c30-ac60-54dd9e3b9d5a admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:51.581147 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1016/2057] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:04:51 2022] PUT /identity/v3/projects/04686fb6d1184bc39b630033fc63d83a/users/ef37d962f6a245da841eaf1959ec0aa7/roles/43106c664e1b40de984c0c62701149c8 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:04:51.592675 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf85236-992f-47d5-98dc-5a26596e0f4c None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:51.593185 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf85236-992f-47d5-98dc-5a26596e0f4c None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:51.593288 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf85236-992f-47d5-98dc-5a26596e0f4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:51.622781 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-fdf85236-992f-47d5-98dc-5a26596e0f4c None None] MFA Rules not processed for user `ef37d962f6a245da841eaf1959ec0aa7`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:04:51.624035 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-fdf85236-992f-47d5-98dc-5a26596e0f4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:51.650111 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:04:51.689631 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1042/2058] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:04:51 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:04:51.719872 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfa4ae8-51e7-4c30-ac60-54dd9e3b9d5a admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:51.721601 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0cfa4ae8-51e7-4c30-ac60-54dd9e3b9d5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:51.780623 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfa4ae8-51e7-4c30-ac60-54dd9e3b9d5a admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:51.784286 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4af981-6e26-487f-860a-1966f7e131c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6cc95a563b941ac92ab283aa0d5a5d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:51.789076 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4af981-6e26-487f-860a-1966f7e131c9 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:51.789076 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4af981-6e26-487f-860a-1966f7e131c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:51.789076 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4af981-6e26-487f-860a-1966f7e131c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:51.793807 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4af981-6e26-487f-860a-1966f7e131c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:51.795216 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4af981-6e26-487f-860a-1966f7e131c9 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:51.804252 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1017/2059] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:04:51 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 10 13:04:51.823775 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf85236-992f-47d5-98dc-5a26596e0f4c None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:51.898954 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf85236-992f-47d5-98dc-5a26596e0f4c None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:51.904239 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d77f6a1-11e2-4697-961a-4296aa773453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:51.905157 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d77f6a1-11e2-4697-961a-4296aa773453 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:51.905376 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d77f6a1-11e2-4697-961a-4296aa773453 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:51.905665 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d77f6a1-11e2-4697-961a-4296aa773453 admin admin] PATH_INFO: `/v3/users/ef37d962f6a245da841eaf1959ec0aa7` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:51.906885 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d77f6a1-11e2-4697-961a-4296aa773453 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef37d962f6a245da841eaf1959ec0aa7)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:51.908898 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d77f6a1-11e2-4697-961a-4296aa773453 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:52.111328 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5d77f6a1-11e2-4697-961a-4296aa773453 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef37d962f6a245da841eaf1959ec0aa7'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:52.128430 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5d77f6a1-11e2-4697-961a-4296aa773453 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef37d962f6a245da841eaf1959ec0aa7'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:52.214048 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5d77f6a1-11e2-4697-961a-4296aa773453 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef37d962f6a245da841eaf1959ec0aa7'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:52.214695 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5d77f6a1-11e2-4697-961a-4296aa773453 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef37d962f6a245da841eaf1959ec0aa7'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:52.227221 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5d77f6a1-11e2-4697-961a-4296aa773453 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef37d962f6a245da841eaf1959ec0aa7'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:52.233886 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1043/2060] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:04:51 2022] DELETE /identity/v3/users/ef37d962f6a245da841eaf1959ec0aa7 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 10 13:04:52.238392 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a0ed87-cfbd-4a95-bdc9-4c4ae2b1b934 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:52.238681 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a0ed87-cfbd-4a95-bdc9-4c4ae2b1b934 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:52.238958 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a0ed87-cfbd-4a95-bdc9-4c4ae2b1b934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:52.266752 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-e1a0ed87-cfbd-4a95-bdc9-4c4ae2b1b934 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:04:52.268284 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e1a0ed87-cfbd-4a95-bdc9-4c4ae2b1b934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:52.269673 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d77f6a1-11e2-4697-961a-4296aa773453 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:52.271156 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5d77f6a1-11e2-4697-961a-4296aa773453 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:52.301272 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:04:52.324206 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:04:52.343117 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d77f6a1-11e2-4697-961a-4296aa773453 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:52.347219 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1018/2061] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:04:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:04:52.348194 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ea4a35-5d74-4d82-b2b2-07bb3fb6d1e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:52.349066 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ea4a35-5d74-4d82-b2b2-07bb3fb6d1e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:52.349395 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ea4a35-5d74-4d82-b2b2-07bb3fb6d1e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:52.349674 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ea4a35-5d74-4d82-b2b2-07bb3fb6d1e9 admin admin] PATH_INFO: `/v3/users/7a06196d87094308b80547966826614d` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:52.350985 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ea4a35-5d74-4d82-b2b2-07bb3fb6d1e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a06196d87094308b80547966826614d)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:52.352930 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ea4a35-5d74-4d82-b2b2-07bb3fb6d1e9 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:52.356814 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a0ed87-cfbd-4a95-bdc9-4c4ae2b1b934 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:52.372492 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e1a0ed87-cfbd-4a95-bdc9-4c4ae2b1b934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:52.413400 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a0ed87-cfbd-4a95-bdc9-4c4ae2b1b934 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:52.417067 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472ed388-36dd-4e31-94de-a5d970377c36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:52.417750 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472ed388-36dd-4e31-94de-a5d970377c36 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:52.418033 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472ed388-36dd-4e31-94de-a5d970377c36 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:52.418347 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472ed388-36dd-4e31-94de-a5d970377c36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:52.428565 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472ed388-36dd-4e31-94de-a5d970377c36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:52.429999 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472ed388-36dd-4e31-94de-a5d970377c36 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:52.439433 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1019/2062] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:04:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 10 13:04:52.539421 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-84ea4a35-5d74-4d82-b2b2-07bb3fb6d1e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a06196d87094308b80547966826614d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:52.540206 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-84ea4a35-5d74-4d82-b2b2-07bb3fb6d1e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a06196d87094308b80547966826614d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:52.553614 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472ed388-36dd-4e31-94de-a5d970377c36 service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:52.555071 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-472ed388-36dd-4e31-94de-a5d970377c36 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:52.567632 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472ed388-36dd-4e31-94de-a5d970377c36 service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:52.581424 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0872fa44-8a26-450f-b7ef-29a158798d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:52.581424 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0872fa44-8a26-450f-b7ef-29a158798d9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:52.581424 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0872fa44-8a26-450f-b7ef-29a158798d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:52.581424 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0872fa44-8a26-450f-b7ef-29a158798d9c admin admin] PATH_INFO: `/v3/users/16eac99a61d34b48b2d4232c1113e780` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:52.581817 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0872fa44-8a26-450f-b7ef-29a158798d9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=16eac99a61d34b48b2d4232c1113e780)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:52.581817 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0872fa44-8a26-450f-b7ef-29a158798d9c admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:52.624108 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-84ea4a35-5d74-4d82-b2b2-07bb3fb6d1e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a06196d87094308b80547966826614d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:52.624971 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-84ea4a35-5d74-4d82-b2b2-07bb3fb6d1e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a06196d87094308b80547966826614d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:52.650736 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-84ea4a35-5d74-4d82-b2b2-07bb3fb6d1e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a06196d87094308b80547966826614d'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:52.660149 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1044/2063] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:04:52 2022] DELETE /identity/v3/users/7a06196d87094308b80547966826614d => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 10 13:04:52.661736 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6e6dee-90b2-4ce8-b15c-974e93d37ecd None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:52.662025 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6e6dee-90b2-4ce8-b15c-974e93d37ecd None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:52.662295 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6e6dee-90b2-4ce8-b15c-974e93d37ecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:52.680315 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-2a6e6dee-90b2-4ce8-b15c-974e93d37ecd None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:04:52.681528 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2a6e6dee-90b2-4ce8-b15c-974e93d37ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:52.740096 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1045/2064] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:04:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 10 13:04:52.755647 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6e6dee-90b2-4ce8-b15c-974e93d37ecd None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:52.757143 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2a6e6dee-90b2-4ce8-b15c-974e93d37ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:52.761608 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-0872fa44-8a26-450f-b7ef-29a158798d9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16eac99a61d34b48b2d4232c1113e780'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:52.769363 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-0872fa44-8a26-450f-b7ef-29a158798d9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16eac99a61d34b48b2d4232c1113e780'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:52.820322 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6e6dee-90b2-4ce8-b15c-974e93d37ecd None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:52.824563 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64439cf-d1f5-42e2-856e-6c6c0ca38a09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:52.825197 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64439cf-d1f5-42e2-856e-6c6c0ca38a09 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:52.825477 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64439cf-d1f5-42e2-856e-6c6c0ca38a09 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:52.825756 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64439cf-d1f5-42e2-856e-6c6c0ca38a09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:52.834658 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64439cf-d1f5-42e2-856e-6c6c0ca38a09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:52.836131 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64439cf-d1f5-42e2-856e-6c6c0ca38a09 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:52.845722 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1046/2065] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:04:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:04:52.865473 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-0872fa44-8a26-450f-b7ef-29a158798d9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16eac99a61d34b48b2d4232c1113e780'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:52.866182 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-0872fa44-8a26-450f-b7ef-29a158798d9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16eac99a61d34b48b2d4232c1113e780'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:52.879623 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64439cf-d1f5-42e2-856e-6c6c0ca38a09 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:52.881921 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d64439cf-d1f5-42e2-856e-6c6c0ca38a09 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:52.883257 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-0872fa44-8a26-450f-b7ef-29a158798d9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16eac99a61d34b48b2d4232c1113e780'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:52.891157 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1020/2066] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:04:52 2022] DELETE /identity/v3/users/16eac99a61d34b48b2d4232c1113e780 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 10 13:04:52.895240 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237cf25d-e217-4b0c-901e-81eefe27106b None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:52.895620 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237cf25d-e217-4b0c-901e-81eefe27106b None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:52.896040 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237cf25d-e217-4b0c-901e-81eefe27106b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:52.902558 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64439cf-d1f5-42e2-856e-6c6c0ca38a09 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:52.916613 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-237cf25d-e217-4b0c-901e-81eefe27106b None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:04:52.918704 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:04:52.919720 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-237cf25d-e217-4b0c-901e-81eefe27106b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:52.967708 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:04:52.980814 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6773f3d2-e48a-4fee-9e75-3abbe36af0ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:52.981673 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6773f3d2-e48a-4fee-9e75-3abbe36af0ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:52.981984 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6773f3d2-e48a-4fee-9e75-3abbe36af0ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:52.982259 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6773f3d2-e48a-4fee-9e75-3abbe36af0ed admin admin] PATH_INFO: `/v3/projects/04686fb6d1184bc39b630033fc63d83a` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:52.983435 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6773f3d2-e48a-4fee-9e75-3abbe36af0ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=04686fb6d1184bc39b630033fc63d83a)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:52.985370 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6773f3d2-e48a-4fee-9e75-3abbe36af0ed admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:52.991689 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1021/2067] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:04:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 10 13:04:53.002174 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237cf25d-e217-4b0c-901e-81eefe27106b None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:53.003518 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-237cf25d-e217-4b0c-901e-81eefe27106b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:53.059895 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237cf25d-e217-4b0c-901e-81eefe27106b None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:53.063808 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13102006-2e99-4508-96ee-a96293026990 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:53.064494 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13102006-2e99-4508-96ee-a96293026990 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:53.064774 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13102006-2e99-4508-96ee-a96293026990 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:53.065056 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13102006-2e99-4508-96ee-a96293026990 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:53.074353 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13102006-2e99-4508-96ee-a96293026990 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:53.075854 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13102006-2e99-4508-96ee-a96293026990 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:53.088249 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1022/2068] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:04:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 10 13:04:53.095487 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6773f3d2-e48a-4fee-9e75-3abbe36af0ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04686fb6d1184bc39b630033fc63d83a', 'request_id': 'req-6773f3d2-e48a-4fee-9e75-3abbe36af0ed'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:53.101278 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6773f3d2-e48a-4fee-9e75-3abbe36af0ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04686fb6d1184bc39b630033fc63d83a', 'request_id': 'req-6773f3d2-e48a-4fee-9e75-3abbe36af0ed'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:53.113655 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6773f3d2-e48a-4fee-9e75-3abbe36af0ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04686fb6d1184bc39b630033fc63d83a', 'request_id': 'req-6773f3d2-e48a-4fee-9e75-3abbe36af0ed'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:53.217489 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6773f3d2-e48a-4fee-9e75-3abbe36af0ed admin admin] The token cache is being invalidate because project 04686fb6d1184bc39b630033fc63d83a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:04:53.217834 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6773f3d2-e48a-4fee-9e75-3abbe36af0ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:53.218960 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6773f3d2-e48a-4fee-9e75-3abbe36af0ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:53.221026 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1047/2069] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:04:52 2022] DELETE /identity/v3/projects/04686fb6d1184bc39b630033fc63d83a => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:04:53.225633 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792a4bf8-8658-4890-85b3-694e217843f9 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:53.225931 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792a4bf8-8658-4890-85b3-694e217843f9 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:53.226206 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792a4bf8-8658-4890-85b3-694e217843f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:53.247621 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-792a4bf8-8658-4890-85b3-694e217843f9 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:04:53.249429 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-792a4bf8-8658-4890-85b3-694e217843f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:53.313718 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1023/2070] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:04:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:04:53.321594 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcfd5d9-02a9-4328-97ac-bf91ae5247c4 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:53.326183 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcfd5d9-02a9-4328-97ac-bf91ae5247c4 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:53.326183 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcfd5d9-02a9-4328-97ac-bf91ae5247c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:53.332072 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792a4bf8-8658-4890-85b3-694e217843f9 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:53.333871 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-792a4bf8-8658-4890-85b3-694e217843f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:53.369023 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792a4bf8-8658-4890-85b3-694e217843f9 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:53.375830 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5375a564-dad5-4c43-bfef-f5db2e3bd361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:53.376268 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-9bcfd5d9-02a9-4328-97ac-bf91ae5247c4 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:04:53.377060 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5375a564-dad5-4c43-bfef-f5db2e3bd361 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:53.377346 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5375a564-dad5-4c43-bfef-f5db2e3bd361 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:53.377715 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5375a564-dad5-4c43-bfef-f5db2e3bd361 admin admin] PATH_INFO: `/v3/projects/264db64881c14d389b1c8f5cdc6cf1fb` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:53.377808 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9bcfd5d9-02a9-4328-97ac-bf91ae5247c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:53.379077 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5375a564-dad5-4c43-bfef-f5db2e3bd361 admin admin] RBAC: Authorizing `identity:delete_project(project_id=264db64881c14d389b1c8f5cdc6cf1fb)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:53.381166 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5375a564-dad5-4c43-bfef-f5db2e3bd361 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:53.388615 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1048/2071] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:04:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:04:53.392842 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcfd5d9-02a9-4328-97ac-bf91ae5247c4 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:53.404688 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcfd5d9-02a9-4328-97ac-bf91ae5247c4 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:53.412313 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb05767-87b9-4c91-a058-121e59e7111c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:53.412313 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb05767-87b9-4c91-a058-121e59e7111c admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:53.412313 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb05767-87b9-4c91-a058-121e59e7111c admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:53.412313 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb05767-87b9-4c91-a058-121e59e7111c admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:53.413046 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb05767-87b9-4c91-a058-121e59e7111c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 13:04:53.413356 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb05767-87b9-4c91-a058-121e59e7111c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:53.414953 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb05767-87b9-4c91-a058-121e59e7111c admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:53.432155 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1049/2072] 10.222.0.27 () {58 vars in 1152 bytes} [Fri Jun 10 13:04:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 13:04:53.433520 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb05767-87b9-4c91-a058-121e59e7111c admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:53.443464 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb05767-87b9-4c91-a058-121e59e7111c admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:53.450757 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c091d044-1c4b-4ebc-8d3d-e489da004e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:53.451671 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c091d044-1c4b-4ebc-8d3d-e489da004e70 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:53.452241 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c091d044-1c4b-4ebc-8d3d-e489da004e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:53.452724 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c091d044-1c4b-4ebc-8d3d-e489da004e70 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:53.453901 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c091d044-1c4b-4ebc-8d3d-e489da004e70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:53.456390 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c091d044-1c4b-4ebc-8d3d-e489da004e70 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:53.499549 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-5375a564-dad5-4c43-bfef-f5db2e3bd361 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '264db64881c14d389b1c8f5cdc6cf1fb', 'request_id': 'req-5375a564-dad5-4c43-bfef-f5db2e3bd361'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:53.504997 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-5375a564-dad5-4c43-bfef-f5db2e3bd361 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '264db64881c14d389b1c8f5cdc6cf1fb', 'request_id': 'req-5375a564-dad5-4c43-bfef-f5db2e3bd361'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:53.511092 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-5375a564-dad5-4c43-bfef-f5db2e3bd361 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '264db64881c14d389b1c8f5cdc6cf1fb', 'request_id': 'req-5375a564-dad5-4c43-bfef-f5db2e3bd361'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:53.545657 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1050/2073] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:04:53 2022] POST /identity/v3/projects => generated 373 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:04:53.549620 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c091d044-1c4b-4ebc-8d3d-e489da004e70 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:53.618971 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-5375a564-dad5-4c43-bfef-f5db2e3bd361 admin admin] The token cache is being invalidate because project 264db64881c14d389b1c8f5cdc6cf1fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:04:53.618971 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-5375a564-dad5-4c43-bfef-f5db2e3bd361 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:53.619983 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-5375a564-dad5-4c43-bfef-f5db2e3bd361 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:53.621096 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:04:53.621900 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1024/2074] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:04:53 2022] DELETE /identity/v3/projects/264db64881c14d389b1c8f5cdc6cf1fb => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:04:53.625885 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5375a564-dad5-4c43-bfef-f5db2e3bd361 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:53.627060 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c091d044-1c4b-4ebc-8d3d-e489da004e70 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:53.627611 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5375a564-dad5-4c43-bfef-f5db2e3bd361 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:53.632245 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a681aa4-b978-4285-9c0d-e35670632c04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:53.633187 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a681aa4-b978-4285-9c0d-e35670632c04 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:53.633477 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a681aa4-b978-4285-9c0d-e35670632c04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:53.633758 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a681aa4-b978-4285-9c0d-e35670632c04 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:53.634453 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a681aa4-b978-4285-9c0d-e35670632c04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:53.636770 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a681aa4-b978-4285-9c0d-e35670632c04 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:53.640698 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:04:53.691952 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5375a564-dad5-4c43-bfef-f5db2e3bd361 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:53.696500 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902e05e5-1aea-43a2-ba66-cd648f4cb775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:53.697359 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902e05e5-1aea-43a2-ba66-cd648f4cb775 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:53.697646 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902e05e5-1aea-43a2-ba66-cd648f4cb775 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:53.697915 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902e05e5-1aea-43a2-ba66-cd648f4cb775 admin admin] PATH_INFO: `/v3/users/742567443239416280ed71f2297b850a` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:53.699097 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902e05e5-1aea-43a2-ba66-cd648f4cb775 admin admin] RBAC: Authorizing `identity:delete_user(user_id=742567443239416280ed71f2297b850a)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:53.701036 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902e05e5-1aea-43a2-ba66-cd648f4cb775 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:53.725041 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1051/2075] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:04:53 2022] POST /identity/v3/users => generated 287 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:04:53.725041 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a681aa4-b978-4285-9c0d-e35670632c04 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:53.725041 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9a681aa4-b978-4285-9c0d-e35670632c04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:53.737821 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a681aa4-b978-4285-9c0d-e35670632c04 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:53.743153 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5caec92-1750-47ca-a384-4fb88208a551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:53.744030 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5caec92-1750-47ca-a384-4fb88208a551 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:53.744316 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5caec92-1750-47ca-a384-4fb88208a551 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:53.744595 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5caec92-1750-47ca-a384-4fb88208a551 admin admin] PATH_INFO: `/v3/projects/3fac54a887fe4da383319fa2c5f65235` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:53.745768 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5caec92-1750-47ca-a384-4fb88208a551 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3fac54a887fe4da383319fa2c5f65235)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:53.747648 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5caec92-1750-47ca-a384-4fb88208a551 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:53.895411 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d5caec92-1750-47ca-a384-4fb88208a551 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3fac54a887fe4da383319fa2c5f65235', 'request_id': 'req-d5caec92-1750-47ca-a384-4fb88208a551'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:53.896137 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-902e05e5-1aea-43a2-ba66-cd648f4cb775 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '742567443239416280ed71f2297b850a'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:53.901628 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d5caec92-1750-47ca-a384-4fb88208a551 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3fac54a887fe4da383319fa2c5f65235', 'request_id': 'req-d5caec92-1750-47ca-a384-4fb88208a551'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:53.906400 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-902e05e5-1aea-43a2-ba66-cd648f4cb775 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '742567443239416280ed71f2297b850a'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:53.908306 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d5caec92-1750-47ca-a384-4fb88208a551 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3fac54a887fe4da383319fa2c5f65235', 'request_id': 'req-d5caec92-1750-47ca-a384-4fb88208a551'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:53.977897 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-902e05e5-1aea-43a2-ba66-cd648f4cb775 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '742567443239416280ed71f2297b850a'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:53.978637 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-902e05e5-1aea-43a2-ba66-cd648f4cb775 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '742567443239416280ed71f2297b850a'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:53.990934 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-902e05e5-1aea-43a2-ba66-cd648f4cb775 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '742567443239416280ed71f2297b850a'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:53.996938 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1025/2076] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:04:53 2022] DELETE /identity/v3/users/742567443239416280ed71f2297b850a => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:04:53.997777 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902e05e5-1aea-43a2-ba66-cd648f4cb775 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:53.999192 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-902e05e5-1aea-43a2-ba66-cd648f4cb775 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:54.036655 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d5caec92-1750-47ca-a384-4fb88208a551 admin admin] The token cache is being invalidate because project 3fac54a887fe4da383319fa2c5f65235 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:04:54.036944 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d5caec92-1750-47ca-a384-4fb88208a551 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:54.037599 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d5caec92-1750-47ca-a384-4fb88208a551 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:54.039156 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1052/2077] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:04:53 2022] DELETE /identity/v3/projects/3fac54a887fe4da383319fa2c5f65235 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:04:54.041982 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5caec92-1750-47ca-a384-4fb88208a551 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:54.041982 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d5caec92-1750-47ca-a384-4fb88208a551 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:54.066050 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902e05e5-1aea-43a2-ba66-cd648f4cb775 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:54.070693 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddc38df-9a76-450b-990e-a9d8851706eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:54.071547 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddc38df-9a76-450b-990e-a9d8851706eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:54.071836 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddc38df-9a76-450b-990e-a9d8851706eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:54.072139 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddc38df-9a76-450b-990e-a9d8851706eb admin admin] PATH_INFO: `/v3/users/478599ffb1684a2abdc89875a56b9b84` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:54.073880 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddc38df-9a76-450b-990e-a9d8851706eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=478599ffb1684a2abdc89875a56b9b84)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:54.075823 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddc38df-9a76-450b-990e-a9d8851706eb admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:54.103304 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5caec92-1750-47ca-a384-4fb88208a551 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:54.107910 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f69c951-a34a-4ff5-88df-ec7e946b78ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:54.108596 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f69c951-a34a-4ff5-88df-ec7e946b78ca admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:54.108863 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f69c951-a34a-4ff5-88df-ec7e946b78ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:54.109170 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f69c951-a34a-4ff5-88df-ec7e946b78ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:54.109894 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f69c951-a34a-4ff5-88df-ec7e946b78ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:54.111469 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f69c951-a34a-4ff5-88df-ec7e946b78ca admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:54.122925 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1053/2078] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:04:54 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:04:54.124154 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cee1ce-b4ac-4177-b66a-e233223b95b4 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:54.124440 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cee1ce-b4ac-4177-b66a-e233223b95b4 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:54.124750 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cee1ce-b4ac-4177-b66a-e233223b95b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:54.141893 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-19cee1ce-b4ac-4177-b66a-e233223b95b4 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:04:54.143129 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-19cee1ce-b4ac-4177-b66a-e233223b95b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:54.151008 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1054/2079] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:04:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:04:54.151747 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cee1ce-b4ac-4177-b66a-e233223b95b4 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:54.152942 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-19cee1ce-b4ac-4177-b66a-e233223b95b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:54.164367 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cee1ce-b4ac-4177-b66a-e233223b95b4 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:54.169145 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809f904f-c195-4be8-b293-eb7a04a0eae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:54.169991 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809f904f-c195-4be8-b293-eb7a04a0eae3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:54.171581 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809f904f-c195-4be8-b293-eb7a04a0eae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:54.171581 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809f904f-c195-4be8-b293-eb7a04a0eae3 admin admin] PATH_INFO: `/v3/users/4082d747bc014ea0a5674de7b2010b14` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:54.171752 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809f904f-c195-4be8-b293-eb7a04a0eae3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4082d747bc014ea0a5674de7b2010b14)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:54.173637 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809f904f-c195-4be8-b293-eb7a04a0eae3 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:54.364910 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-5ddc38df-9a76-450b-990e-a9d8851706eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '478599ffb1684a2abdc89875a56b9b84'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:54.372192 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-5ddc38df-9a76-450b-990e-a9d8851706eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '478599ffb1684a2abdc89875a56b9b84'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:54.421378 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-809f904f-c195-4be8-b293-eb7a04a0eae3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4082d747bc014ea0a5674de7b2010b14'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:54.426047 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=82,keystone:INSERT=4,keystone:DELETE=14 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:04:54.428898 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-809f904f-c195-4be8-b293-eb7a04a0eae3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4082d747bc014ea0a5674de7b2010b14'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:54.436935 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:04:54.438185 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-5ddc38df-9a76-450b-990e-a9d8851706eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '478599ffb1684a2abdc89875a56b9b84'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:54.439916 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-5ddc38df-9a76-450b-990e-a9d8851706eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '478599ffb1684a2abdc89875a56b9b84'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:54.451572 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-5ddc38df-9a76-450b-990e-a9d8851706eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '478599ffb1684a2abdc89875a56b9b84'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:54.458386 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1026/2080] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:04:53 2022] DELETE /identity/v3/users/478599ffb1684a2abdc89875a56b9b84 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:04:54.459296 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddc38df-9a76-450b-990e-a9d8851706eb admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:54.460815 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5ddc38df-9a76-450b-990e-a9d8851706eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:54.536447 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddc38df-9a76-450b-990e-a9d8851706eb admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:54.543553 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-809f904f-c195-4be8-b293-eb7a04a0eae3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4082d747bc014ea0a5674de7b2010b14'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:54.544425 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-809f904f-c195-4be8-b293-eb7a04a0eae3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4082d747bc014ea0a5674de7b2010b14'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:54.549159 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff6667e-7539-40f3-94d8-fbe6ea98c6e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:54.549159 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff6667e-7539-40f3-94d8-fbe6ea98c6e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:54.549159 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff6667e-7539-40f3-94d8-fbe6ea98c6e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:54.549159 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff6667e-7539-40f3-94d8-fbe6ea98c6e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:54.549555 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff6667e-7539-40f3-94d8-fbe6ea98c6e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:54.550150 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff6667e-7539-40f3-94d8-fbe6ea98c6e0 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:54.563134 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-809f904f-c195-4be8-b293-eb7a04a0eae3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4082d747bc014ea0a5674de7b2010b14'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:54.564655 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1027/2081] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:04:54 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:04:54.565902 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff6667e-7539-40f3-94d8-fbe6ea98c6e0 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:54.568478 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-fff6667e-7539-40f3-94d8-fbe6ea98c6e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:54.573613 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1055/2082] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:04:54 2022] DELETE /identity/v3/users/4082d747bc014ea0a5674de7b2010b14 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:04:54.577756 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809f904f-c195-4be8-b293-eb7a04a0eae3 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:54.577756 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-809f904f-c195-4be8-b293-eb7a04a0eae3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:54.646301 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff6667e-7539-40f3-94d8-fbe6ea98c6e0 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:54.648685 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809f904f-c195-4be8-b293-eb7a04a0eae3 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:54.655181 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2d18c5-71a7-4293-abe7-213fe1b7080f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:54.656161 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2d18c5-71a7-4293-abe7-213fe1b7080f admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:54.656474 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2d18c5-71a7-4293-abe7-213fe1b7080f admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:54.656867 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2d18c5-71a7-4293-abe7-213fe1b7080f admin admin] PATH_INFO: `/v3/projects/16dc27d7792c4f92894ed9f677880bc3/users/3dcd9c7df0974b068d6017e85dc85a90/roles/43106c664e1b40de984c0c62701149c8` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:54.675530 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2d18c5-71a7-4293-abe7-213fe1b7080f admin admin] RBAC: Authorizing `identity:create_grant(project_id=16dc27d7792c4f92894ed9f677880bc3, user_id=3dcd9c7df0974b068d6017e85dc85a90, role_id=43106c664e1b40de984c0c62701149c8)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:54.678222 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2d18c5-71a7-4293-abe7-213fe1b7080f admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:54.700278 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2015b109-f574-4a9b-aa48-36765d5a53b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:54.701017 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2015b109-f574-4a9b-aa48-36765d5a53b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:54.701367 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2015b109-f574-4a9b-aa48-36765d5a53b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:54.701650 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2015b109-f574-4a9b-aa48-36765d5a53b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:54.703543 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-2015b109-f574-4a9b-aa48-36765d5a53b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:54.715253 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2015b109-f574-4a9b-aa48-36765d5a53b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:54.716799 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2015b109-f574-4a9b-aa48-36765d5a53b9 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:54.727438 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1028/2083] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:04:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:04:54.751742 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1056/2084] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:04:54 2022] PUT /identity/v3/projects/16dc27d7792c4f92894ed9f677880bc3/users/3dcd9c7df0974b068d6017e85dc85a90/roles/43106c664e1b40de984c0c62701149c8 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:04:54.763753 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0076b790-5801-441b-91cb-ad8a22352be3 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:54.764143 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0076b790-5801-441b-91cb-ad8a22352be3 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:54.764490 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0076b790-5801-441b-91cb-ad8a22352be3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:54.798255 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-0076b790-5801-441b-91cb-ad8a22352be3 None None] MFA Rules not processed for user `3dcd9c7df0974b068d6017e85dc85a90`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:04:54.799854 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0076b790-5801-441b-91cb-ad8a22352be3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:54.876606 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1029/2085] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:04:54 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 10 13:04:54.888473 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2d18c5-71a7-4293-abe7-213fe1b7080f admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:54.890421 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-4c2d18c5-71a7-4293-abe7-213fe1b7080f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:54.946150 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0076b790-5801-441b-91cb-ad8a22352be3 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:54.946150 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0076b790-5801-441b-91cb-ad8a22352be3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:54.953612 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2d18c5-71a7-4293-abe7-213fe1b7080f admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:54.957719 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13d02c9-7278-4425-b311-b8eca9526f9e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6cc95a563b941ac92ab283aa0d5a5d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:54.958440 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13d02c9-7278-4425-b311-b8eca9526f9e service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:54.958709 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13d02c9-7278-4425-b311-b8eca9526f9e service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:54.958984 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13d02c9-7278-4425-b311-b8eca9526f9e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:54.974823 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13d02c9-7278-4425-b311-b8eca9526f9e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:54.976382 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13d02c9-7278-4425-b311-b8eca9526f9e service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:54.980672 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:04:54.991516 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1057/2086] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:04:54 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:04:55.058402 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0076b790-5801-441b-91cb-ad8a22352be3 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:55.065547 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e63dfd3-0ed8-454a-bdce-ac1f063f7c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:55.067391 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63dfd3-0ed8-454a-bdce-ac1f063f7c54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:55.067894 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63dfd3-0ed8-454a-bdce-ac1f063f7c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:55.068186 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e63dfd3-0ed8-454a-bdce-ac1f063f7c54 admin admin] PATH_INFO: `/v3/projects/545b53cb243d4428a207594c827f9931` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:55.069555 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e63dfd3-0ed8-454a-bdce-ac1f063f7c54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=545b53cb243d4428a207594c827f9931)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:55.071476 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e63dfd3-0ed8-454a-bdce-ac1f063f7c54 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:55.081377 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13d02c9-7278-4425-b311-b8eca9526f9e service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:55.084006 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c13d02c9-7278-4425-b311-b8eca9526f9e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:55.105353 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13d02c9-7278-4425-b311-b8eca9526f9e service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:55.112796 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fc752e-5f70-4f16-aa71-571a33509959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:55.113890 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fc752e-5f70-4f16-aa71-571a33509959 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:55.114405 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fc752e-5f70-4f16-aa71-571a33509959 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:55.114655 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fc752e-5f70-4f16-aa71-571a33509959 admin admin] PATH_INFO: `/v3/projects/5726ef9ae4eb41419b669f122be6729f` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:55.115913 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fc752e-5f70-4f16-aa71-571a33509959 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5726ef9ae4eb41419b669f122be6729f)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:55.117835 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fc752e-5f70-4f16-aa71-571a33509959 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:55.219172 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-8e63dfd3-0ed8-454a-bdce-ac1f063f7c54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '545b53cb243d4428a207594c827f9931', 'request_id': 'req-8e63dfd3-0ed8-454a-bdce-ac1f063f7c54'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:55.223019 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-74fc752e-5f70-4f16-aa71-571a33509959 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5726ef9ae4eb41419b669f122be6729f', 'request_id': 'req-74fc752e-5f70-4f16-aa71-571a33509959'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:55.224233 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:04:55.225161 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-8e63dfd3-0ed8-454a-bdce-ac1f063f7c54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '545b53cb243d4428a207594c827f9931', 'request_id': 'req-8e63dfd3-0ed8-454a-bdce-ac1f063f7c54'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:55.230114 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-74fc752e-5f70-4f16-aa71-571a33509959 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5726ef9ae4eb41419b669f122be6729f', 'request_id': 'req-74fc752e-5f70-4f16-aa71-571a33509959'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:55.233486 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-8e63dfd3-0ed8-454a-bdce-ac1f063f7c54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '545b53cb243d4428a207594c827f9931', 'request_id': 'req-8e63dfd3-0ed8-454a-bdce-ac1f063f7c54'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:55.236208 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-74fc752e-5f70-4f16-aa71-571a33509959 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5726ef9ae4eb41419b669f122be6729f', 'request_id': 'req-74fc752e-5f70-4f16-aa71-571a33509959'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:55.312666 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-8e63dfd3-0ed8-454a-bdce-ac1f063f7c54 admin admin] The token cache is being invalidate because project 545b53cb243d4428a207594c827f9931 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:04:55.312666 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-8e63dfd3-0ed8-454a-bdce-ac1f063f7c54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:55.312666 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-8e63dfd3-0ed8-454a-bdce-ac1f063f7c54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:55.312666 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1030/2087] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:04:54 2022] DELETE /identity/v3/projects/545b53cb243d4428a207594c827f9931 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 10 13:04:55.312666 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e63dfd3-0ed8-454a-bdce-ac1f063f7c54 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:55.313354 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-8e63dfd3-0ed8-454a-bdce-ac1f063f7c54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:55.374996 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e63dfd3-0ed8-454a-bdce-ac1f063f7c54 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:55.429534 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d51d5b1-4fcc-4988-b938-89344444a4f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:55.430415 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d51d5b1-4fcc-4988-b938-89344444a4f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:55.430697 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d51d5b1-4fcc-4988-b938-89344444a4f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:55.430970 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d51d5b1-4fcc-4988-b938-89344444a4f2 admin admin] PATH_INFO: `/v3/projects/72116263e3af4375976c608558f8967b` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:55.432271 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d51d5b1-4fcc-4988-b938-89344444a4f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=72116263e3af4375976c608558f8967b)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:55.434279 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d51d5b1-4fcc-4988-b938-89344444a4f2 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:55.454245 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-74fc752e-5f70-4f16-aa71-571a33509959 admin admin] The token cache is being invalidate because project 5726ef9ae4eb41419b669f122be6729f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:04:55.454699 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-74fc752e-5f70-4f16-aa71-571a33509959 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:55.455439 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-74fc752e-5f70-4f16-aa71-571a33509959 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:55.460636 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1058/2088] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:04:55 2022] DELETE /identity/v3/projects/5726ef9ae4eb41419b669f122be6729f => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 10 13:04:55.460636 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e12316-b4af-413f-a4df-5d3d93417370 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:55.460636 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e12316-b4af-413f-a4df-5d3d93417370 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:55.460636 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e12316-b4af-413f-a4df-5d3d93417370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:55.488272 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-d0e12316-b4af-413f-a4df-5d3d93417370 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:04:55.489546 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d0e12316-b4af-413f-a4df-5d3d93417370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:55.498500 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1059/2089] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:04:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:04:55.499734 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bd3471-5714-4e47-b446-5c3837de3889 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:55.500055 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bd3471-5714-4e47-b446-5c3837de3889 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:55.500331 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bd3471-5714-4e47-b446-5c3837de3889 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:55.519629 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-f2bd3471-5714-4e47-b446-5c3837de3889 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:04:55.520881 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f2bd3471-5714-4e47-b446-5c3837de3889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:55.529589 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1060/2090] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:04:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:04:55.530779 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be76e71-5e55-4c3e-b1ef-974e77f6aeda None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:55.531066 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be76e71-5e55-4c3e-b1ef-974e77f6aeda None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:55.531347 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be76e71-5e55-4c3e-b1ef-974e77f6aeda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:55.572926 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2d51d5b1-4fcc-4988-b938-89344444a4f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72116263e3af4375976c608558f8967b', 'request_id': 'req-2d51d5b1-4fcc-4988-b938-89344444a4f2'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:55.574049 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-1be76e71-5e55-4c3e-b1ef-974e77f6aeda None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:04:55.576167 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1be76e71-5e55-4c3e-b1ef-974e77f6aeda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:55.580806 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2d51d5b1-4fcc-4988-b938-89344444a4f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72116263e3af4375976c608558f8967b', 'request_id': 'req-2d51d5b1-4fcc-4988-b938-89344444a4f2'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:55.593173 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2d51d5b1-4fcc-4988-b938-89344444a4f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72116263e3af4375976c608558f8967b', 'request_id': 'req-2d51d5b1-4fcc-4988-b938-89344444a4f2'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:55.649377 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1061/2091] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:04:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:04:55.650714 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d676d6c5-6a4d-4225-9f3d-3a5dc63d7af7 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:55.651160 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d676d6c5-6a4d-4225-9f3d-3a5dc63d7af7 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:55.651444 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d676d6c5-6a4d-4225-9f3d-3a5dc63d7af7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:55.673989 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-d676d6c5-6a4d-4225-9f3d-3a5dc63d7af7 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:04:55.675192 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d676d6c5-6a4d-4225-9f3d-3a5dc63d7af7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:55.684549 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1062/2092] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:04:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:04:55.685166 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d676d6c5-6a4d-4225-9f3d-3a5dc63d7af7 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:55.697048 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2d51d5b1-4fcc-4988-b938-89344444a4f2 admin admin] The token cache is being invalidate because project 72116263e3af4375976c608558f8967b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:04:55.698456 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2d51d5b1-4fcc-4988-b938-89344444a4f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:55.700239 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2d51d5b1-4fcc-4988-b938-89344444a4f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:04:55.701097 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d676d6c5-6a4d-4225-9f3d-3a5dc63d7af7 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:55.703137 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1031/2093] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:04:55 2022] DELETE /identity/v3/projects/72116263e3af4375976c608558f8967b => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:04:55.704848 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d51d5b1-4fcc-4988-b938-89344444a4f2 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:55.706440 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7a5e23-e9db-4732-854e-4ba98b21b2ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:55.706811 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-2d51d5b1-4fcc-4988-b938-89344444a4f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:55.707359 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7a5e23-e9db-4732-854e-4ba98b21b2ca admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:55.707948 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7a5e23-e9db-4732-854e-4ba98b21b2ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:55.708309 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7a5e23-e9db-4732-854e-4ba98b21b2ca admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:55.709645 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7a5e23-e9db-4732-854e-4ba98b21b2ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 13:04:55.710036 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7a5e23-e9db-4732-854e-4ba98b21b2ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:55.712155 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7a5e23-e9db-4732-854e-4ba98b21b2ca admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:55.730966 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d51d5b1-4fcc-4988-b938-89344444a4f2 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:55.735904 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1063/2094] 10.222.0.27 () {58 vars in 1152 bytes} [Fri Jun 10 13:04:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 13:04:55.735904 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83711c0c-0fe2-43db-a2fe-e37424a2a614 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:55.736138 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ad6591-618f-46e9-88f7-c45028741981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:55.736250 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83711c0c-0fe2-43db-a2fe-e37424a2a614 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:55.736250 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83711c0c-0fe2-43db-a2fe-e37424a2a614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:55.736391 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ad6591-618f-46e9-88f7-c45028741981 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:55.736651 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ad6591-618f-46e9-88f7-c45028741981 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:55.736969 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ad6591-618f-46e9-88f7-c45028741981 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:55.737661 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ad6591-618f-46e9-88f7-c45028741981 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 13:04:55.737971 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ad6591-618f-46e9-88f7-c45028741981 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:55.739622 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ad6591-618f-46e9-88f7-c45028741981 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:55.752477 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1032/2095] 10.222.0.27 () {58 vars in 1152 bytes} [Fri Jun 10 13:04:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 13:04:55.755370 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ad6591-618f-46e9-88f7-c45028741981 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:55.758887 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d5ad6591-618f-46e9-88f7-c45028741981 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:55.759265 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-83711c0c-0fe2-43db-a2fe-e37424a2a614 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:04:55.761351 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-83711c0c-0fe2-43db-a2fe-e37424a2a614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:55.769931 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1064/2096] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:04:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:04:55.778198 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83711c0c-0fe2-43db-a2fe-e37424a2a614 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:55.782314 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ad6591-618f-46e9-88f7-c45028741981 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:55.787629 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83711c0c-0fe2-43db-a2fe-e37424a2a614 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:55.790049 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c0f0eb-1db5-4b2e-aaa4-f5c41b659ebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:55.791203 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c0f0eb-1db5-4b2e-aaa4-f5c41b659ebe admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:55.791637 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c0f0eb-1db5-4b2e-aaa4-f5c41b659ebe admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:55.792108 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c0f0eb-1db5-4b2e-aaa4-f5c41b659ebe admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:55.792941 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d67084f-c193-4701-8ec0-00a47c62884d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:55.793081 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c0f0eb-1db5-4b2e-aaa4-f5c41b659ebe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:55.793616 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d67084f-c193-4701-8ec0-00a47c62884d admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:55.793902 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d67084f-c193-4701-8ec0-00a47c62884d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:55.794180 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d67084f-c193-4701-8ec0-00a47c62884d admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:55.794857 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c0f0eb-1db5-4b2e-aaa4-f5c41b659ebe admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:55.795222 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d67084f-c193-4701-8ec0-00a47c62884d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:55.797154 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d67084f-c193-4701-8ec0-00a47c62884d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:55.825064 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:04:55.882035 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1065/2097] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:04:55 2022] POST /identity/v3/projects => generated 375 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:04:55.882035 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a0609e-e47f-45ac-bee1-c15e95f5242e None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:55.882035 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a0609e-e47f-45ac-bee1-c15e95f5242e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:55.882035 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a0609e-e47f-45ac-bee1-c15e95f5242e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:55.900701 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-19a0609e-e47f-45ac-bee1-c15e95f5242e None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:04:55.906441 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-19a0609e-e47f-45ac-bee1-c15e95f5242e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:55.906574 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1033/2098] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:04:55 2022] POST /identity/v3/projects => generated 373 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:04:55.906574 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c0f0eb-1db5-4b2e-aaa4-f5c41b659ebe admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:55.964029 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1066/2099] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:04:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:04:55.964917 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a0609e-e47f-45ac-bee1-c15e95f5242e None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:55.965920 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c0f0eb-1db5-4b2e-aaa4-f5c41b659ebe admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:55.971193 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b570076-a504-4e7f-8418-a9e867277d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:55.971945 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b570076-a504-4e7f-8418-a9e867277d7a admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:55.972234 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b570076-a504-4e7f-8418-a9e867277d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:55.972515 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b570076-a504-4e7f-8418-a9e867277d7a admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:55.973260 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b570076-a504-4e7f-8418-a9e867277d7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:55.974415 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a0609e-e47f-45ac-bee1-c15e95f5242e None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:55.975325 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b570076-a504-4e7f-8418-a9e867277d7a admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:55.979573 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4ac228-835f-4de2-afbf-f74e779d29f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:55.980245 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4ac228-835f-4de2-afbf-f74e779d29f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:55.980525 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4ac228-835f-4de2-afbf-f74e779d29f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:55.980800 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4ac228-835f-4de2-afbf-f74e779d29f5 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:55.981483 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4ac228-835f-4de2-afbf-f74e779d29f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:55.983392 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4ac228-835f-4de2-afbf-f74e779d29f5 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:56.041810 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1034/2100] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:04:55 2022] POST /identity/v3/users => generated 288 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:04:56.042828 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b570076-a504-4e7f-8418-a9e867277d7a admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:56.052410 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b570076-a504-4e7f-8418-a9e867277d7a admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:56.058026 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2346130d-13d5-4a88-a4c0-24f8f8c35253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:56.059163 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2346130d-13d5-4a88-a4c0-24f8f8c35253 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:56.060068 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2346130d-13d5-4a88-a4c0-24f8f8c35253 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:56.062123 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2346130d-13d5-4a88-a4c0-24f8f8c35253 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:56.062123 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2346130d-13d5-4a88-a4c0-24f8f8c35253 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 13:04:56.062123 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2346130d-13d5-4a88-a4c0-24f8f8c35253 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:56.063168 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2346130d-13d5-4a88-a4c0-24f8f8c35253 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:56.067408 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1067/2101] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:04:55 2022] POST /identity/v3/users => generated 287 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:04:56.067408 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4ac228-835f-4de2-afbf-f74e779d29f5 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:56.074721 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4ac228-835f-4de2-afbf-f74e779d29f5 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:56.076051 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1035/2102] 10.222.0.27 () {58 vars in 1152 bytes} [Fri Jun 10 13:04:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 13:04:56.076791 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2346130d-13d5-4a88-a4c0-24f8f8c35253 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:56.082666 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dffb0c3-4f93-4ef2-9f77-ad87dfbae8b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:56.083319 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dffb0c3-4f93-4ef2-9f77-ad87dfbae8b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:56.083602 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dffb0c3-4f93-4ef2-9f77-ad87dfbae8b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:56.089872 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dffb0c3-4f93-4ef2-9f77-ad87dfbae8b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:56.089872 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dffb0c3-4f93-4ef2-9f77-ad87dfbae8b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:56.089872 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dffb0c3-4f93-4ef2-9f77-ad87dfbae8b1 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:56.090145 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2346130d-13d5-4a88-a4c0-24f8f8c35253 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:56.091842 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ff5d20-b14d-4ed4-a6b2-8aea4c6b079b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:56.092518 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ff5d20-b14d-4ed4-a6b2-8aea4c6b079b admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:56.092807 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ff5d20-b14d-4ed4-a6b2-8aea4c6b079b admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:56.093081 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ff5d20-b14d-4ed4-a6b2-8aea4c6b079b admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:56.093813 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ff5d20-b14d-4ed4-a6b2-8aea4c6b079b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:56.095384 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ff5d20-b14d-4ed4-a6b2-8aea4c6b079b admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:56.096757 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1068/2103] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:04:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:04:56.097313 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dffb0c3-4f93-4ef2-9f77-ad87dfbae8b1 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:56.109671 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1036/2104] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:04:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:04:56.109671 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ff5d20-b14d-4ed4-a6b2-8aea4c6b079b admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:56.109908 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dffb0c3-4f93-4ef2-9f77-ad87dfbae8b1 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:56.114591 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eec1d60-9c51-4d21-b534-c385de17c746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:56.115249 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eec1d60-9c51-4d21-b534-c385de17c746 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:56.115533 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eec1d60-9c51-4d21-b534-c385de17c746 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:56.115833 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eec1d60-9c51-4d21-b534-c385de17c746 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:56.116548 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eec1d60-9c51-4d21-b534-c385de17c746 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:56.118548 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eec1d60-9c51-4d21-b534-c385de17c746 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:56.119075 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ff5d20-b14d-4ed4-a6b2-8aea4c6b079b admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:56.124335 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3821ebe7-e135-42e3-8ecb-043f0c1681c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:56.124978 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3821ebe7-e135-42e3-8ecb-043f0c1681c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:56.125283 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3821ebe7-e135-42e3-8ecb-043f0c1681c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:56.125601 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3821ebe7-e135-42e3-8ecb-043f0c1681c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:56.126332 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3821ebe7-e135-42e3-8ecb-043f0c1681c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:56.127947 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3821ebe7-e135-42e3-8ecb-043f0c1681c2 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:56.134650 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:DELETE=3,keystone:SELECT=92,keystone:INSERT=5 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:04:56.138172 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1037/2105] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:04:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:04:56.138960 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3821ebe7-e135-42e3-8ecb-043f0c1681c2 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:56.149371 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3821ebe7-e135-42e3-8ecb-043f0c1681c2 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:56.153999 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ca5e72-613e-481d-b9b9-643ef30dd744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:56.154682 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ca5e72-613e-481d-b9b9-643ef30dd744 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:56.159782 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ca5e72-613e-481d-b9b9-643ef30dd744 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:56.159782 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ca5e72-613e-481d-b9b9-643ef30dd744 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:56.159782 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ca5e72-613e-481d-b9b9-643ef30dd744 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:56.159782 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ca5e72-613e-481d-b9b9-643ef30dd744 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:56.168356 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1038/2106] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:04:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:04:56.169307 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ca5e72-613e-481d-b9b9-643ef30dd744 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:56.178813 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ca5e72-613e-481d-b9b9-643ef30dd744 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:56.184252 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4ba4b4-bd70-4629-924e-5bb5496db349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:56.185082 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4ba4b4-bd70-4629-924e-5bb5496db349 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:56.185357 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4ba4b4-bd70-4629-924e-5bb5496db349 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:56.185627 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4ba4b4-bd70-4629-924e-5bb5496db349 admin admin] PATH_INFO: `/v3/projects/11f9adc51a60485d8c2a745d92782d76/users/785ada42968a40a88abbbdb69de55225/roles/43106c664e1b40de984c0c62701149c8` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:56.200184 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4ba4b4-bd70-4629-924e-5bb5496db349 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11f9adc51a60485d8c2a745d92782d76, user_id=785ada42968a40a88abbbdb69de55225, role_id=43106c664e1b40de984c0c62701149c8)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:56.202868 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4ba4b4-bd70-4629-924e-5bb5496db349 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:56.263440 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1069/2107] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:04:56 2022] POST /identity/v3/projects => generated 373 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:04:56.263440 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eec1d60-9c51-4d21-b534-c385de17c746 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:56.287937 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1039/2108] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:04:56 2022] PUT /identity/v3/projects/11f9adc51a60485d8c2a745d92782d76/users/785ada42968a40a88abbbdb69de55225/roles/43106c664e1b40de984c0c62701149c8 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:04:56.288738 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4ba4b4-bd70-4629-924e-5bb5496db349 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:56.311519 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:04:56.329364 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eec1d60-9c51-4d21-b534-c385de17c746 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:56.352096 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4ba4b4-bd70-4629-924e-5bb5496db349 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:56.357293 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd7a0fd-7e50-41b6-adae-ecc1973e77f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:56.357976 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd7a0fd-7e50-41b6-adae-ecc1973e77f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:56.358260 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd7a0fd-7e50-41b6-adae-ecc1973e77f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:56.358544 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd7a0fd-7e50-41b6-adae-ecc1973e77f2 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:56.359363 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd7a0fd-7e50-41b6-adae-ecc1973e77f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:56.362456 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd7a0fd-7e50-41b6-adae-ecc1973e77f2 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:56.385065 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450b2845-c3b2-4db1-af2f-d291e8bfc8c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:56.385951 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450b2845-c3b2-4db1-af2f-d291e8bfc8c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:56.386233 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450b2845-c3b2-4db1-af2f-d291e8bfc8c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:56.386508 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450b2845-c3b2-4db1-af2f-d291e8bfc8c6 admin admin] PATH_INFO: `/v3/projects/bae19a1e72e34b5f9527462fe749b39e/users/e594b4b1ce424b6cbf571655a963cfc8/roles/43106c664e1b40de984c0c62701149c8` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:56.401006 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450b2845-c3b2-4db1-af2f-d291e8bfc8c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bae19a1e72e34b5f9527462fe749b39e, user_id=e594b4b1ce424b6cbf571655a963cfc8, role_id=43106c664e1b40de984c0c62701149c8)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:56.403620 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450b2845-c3b2-4db1-af2f-d291e8bfc8c6 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:56.424661 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1040/2109] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:04:56 2022] POST /identity/v3/users => generated 287 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:04:56.425925 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2e94b7-e7e1-4454-9c85-7b3e3b8a00be None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:56.426213 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2e94b7-e7e1-4454-9c85-7b3e3b8a00be None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:56.426487 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2e94b7-e7e1-4454-9c85-7b3e3b8a00be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:56.452411 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-1a2e94b7-e7e1-4454-9c85-7b3e3b8a00be None None] MFA Rules not processed for user `785ada42968a40a88abbbdb69de55225`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:04:56.453609 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-1a2e94b7-e7e1-4454-9c85-7b3e3b8a00be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:56.457801 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1070/2110] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:04:56 2022] PUT /identity/v3/projects/bae19a1e72e34b5f9527462fe749b39e/users/e594b4b1ce424b6cbf571655a963cfc8/roles/43106c664e1b40de984c0c62701149c8 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:04:56.458414 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450b2845-c3b2-4db1-af2f-d291e8bfc8c6 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:56.482358 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:04:56.519402 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1041/2111] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:04:56 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:04:56.520651 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cfaa08-a791-45b1-a5e3-9c61d173b46f None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:56.521082 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cfaa08-a791-45b1-a5e3-9c61d173b46f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:56.521368 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cfaa08-a791-45b1-a5e3-9c61d173b46f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:56.526444 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:04:56.530247 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450b2845-c3b2-4db1-af2f-d291e8bfc8c6 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:56.535229 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d6460e-057a-4764-a6be-fc1932db1061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:56.541912 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d6460e-057a-4764-a6be-fc1932db1061 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:56.541912 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d6460e-057a-4764-a6be-fc1932db1061 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:56.541912 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d6460e-057a-4764-a6be-fc1932db1061 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:56.541912 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d6460e-057a-4764-a6be-fc1932db1061 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:56.541912 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d6460e-057a-4764-a6be-fc1932db1061 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:56.548027 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-09cfaa08-a791-45b1-a5e3-9c61d173b46f None None] MFA Rules not processed for user `e594b4b1ce424b6cbf571655a963cfc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:04:56.549220 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-09cfaa08-a791-45b1-a5e3-9c61d173b46f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:56.549523 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1071/2112] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:04:56 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:04:56.553017 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d6460e-057a-4764-a6be-fc1932db1061 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:56.561558 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d6460e-057a-4764-a6be-fc1932db1061 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:56.566155 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fcf8bb-072b-4d2a-8325-99f71fa8dece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:56.566781 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fcf8bb-072b-4d2a-8325-99f71fa8dece admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:56.567062 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fcf8bb-072b-4d2a-8325-99f71fa8dece admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:56.569593 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fcf8bb-072b-4d2a-8325-99f71fa8dece admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:56.569593 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fcf8bb-072b-4d2a-8325-99f71fa8dece admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:56.569873 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fcf8bb-072b-4d2a-8325-99f71fa8dece admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:56.613001 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1042/2113] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:04:56 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:04:56.613989 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cfaa08-a791-45b1-a5e3-9c61d173b46f None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:56.624500 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cfaa08-a791-45b1-a5e3-9c61d173b46f None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:56.629153 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e360b5-5d49-48d2-8202-c4f1aba4b000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:56.629800 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e360b5-5d49-48d2-8202-c4f1aba4b000 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:56.630082 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e360b5-5d49-48d2-8202-c4f1aba4b000 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:56.630356 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e360b5-5d49-48d2-8202-c4f1aba4b000 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:56.631074 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e360b5-5d49-48d2-8202-c4f1aba4b000 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:56.632690 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e360b5-5d49-48d2-8202-c4f1aba4b000 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:56.642351 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1043/2114] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:04:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:04:56.643102 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e360b5-5d49-48d2-8202-c4f1aba4b000 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:56.644744 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-11e360b5-5d49-48d2-8202-c4f1aba4b000 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:56.676687 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1072/2115] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:04:56 2022] POST /identity/v3/projects => generated 373 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:04:56.676687 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fcf8bb-072b-4d2a-8325-99f71fa8dece admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:56.702051 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e360b5-5d49-48d2-8202-c4f1aba4b000 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:56.734252 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fcf8bb-072b-4d2a-8325-99f71fa8dece admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:56.740099 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161a0e61-2a9c-4ac8-86f8-11aaaabb9ee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:56.740996 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161a0e61-2a9c-4ac8-86f8-11aaaabb9ee8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:56.741282 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161a0e61-2a9c-4ac8-86f8-11aaaabb9ee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:56.741573 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161a0e61-2a9c-4ac8-86f8-11aaaabb9ee8 admin admin] PATH_INFO: `/v3/projects/b61776123cea47be842ccea6c79dfc12/users/a72b5e893615479ea3fe1838d07496ba/roles/43106c664e1b40de984c0c62701149c8` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:56.754864 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e022b47-a9f9-4776-92da-889ac26cb63f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6cc95a563b941ac92ab283aa0d5a5d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:56.755519 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e022b47-a9f9-4776-92da-889ac26cb63f service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:56.755804 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e022b47-a9f9-4776-92da-889ac26cb63f service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:56.756114 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e022b47-a9f9-4776-92da-889ac26cb63f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:56.758034 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161a0e61-2a9c-4ac8-86f8-11aaaabb9ee8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b61776123cea47be842ccea6c79dfc12, user_id=a72b5e893615479ea3fe1838d07496ba, role_id=43106c664e1b40de984c0c62701149c8)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:56.761295 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161a0e61-2a9c-4ac8-86f8-11aaaabb9ee8 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:56.766957 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:04:56.814194 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e022b47-a9f9-4776-92da-889ac26cb63f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:56.815674 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e022b47-a9f9-4776-92da-889ac26cb63f service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:56.828196 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1044/2116] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:04:56 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:04:56.828196 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e022b47-a9f9-4776-92da-889ac26cb63f service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:56.837719 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e022b47-a9f9-4776-92da-889ac26cb63f service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:56.843308 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30284c33-2ce9-417b-bc86-0dde3d611cc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:56.844010 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30284c33-2ce9-417b-bc86-0dde3d611cc1 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:56.844299 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30284c33-2ce9-417b-bc86-0dde3d611cc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:56.844579 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30284c33-2ce9-417b-bc86-0dde3d611cc1 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:56.845526 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30284c33-2ce9-417b-bc86-0dde3d611cc1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:56.847205 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30284c33-2ce9-417b-bc86-0dde3d611cc1 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:56.873029 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1073/2117] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:04:56 2022] PUT /identity/v3/projects/b61776123cea47be842ccea6c79dfc12/users/a72b5e893615479ea3fe1838d07496ba/roles/43106c664e1b40de984c0c62701149c8 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:04:56.876068 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58701d23-76a2-44e8-842a-4e405e1ba8d2 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:56.876452 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58701d23-76a2-44e8-842a-4e405e1ba8d2 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:56.876907 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58701d23-76a2-44e8-842a-4e405e1ba8d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:56.899121 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-58701d23-76a2-44e8-842a-4e405e1ba8d2 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:04:56.901304 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-58701d23-76a2-44e8-842a-4e405e1ba8d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:56.949769 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1045/2118] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:04:56 2022] POST /identity/v3/users => generated 293 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:04:56.951416 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30284c33-2ce9-417b-bc86-0dde3d611cc1 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:56.965657 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1074/2119] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:04:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 10 13:04:56.967411 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30284c33-2ce9-417b-bc86-0dde3d611cc1 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:56.972827 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65210b9c-f3b3-48a2-b187-644c8ec77813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:56.973513 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65210b9c-f3b3-48a2-b187-644c8ec77813 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:56.973805 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65210b9c-f3b3-48a2-b187-644c8ec77813 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:56.974080 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65210b9c-f3b3-48a2-b187-644c8ec77813 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:56.974812 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65210b9c-f3b3-48a2-b187-644c8ec77813 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:56.976446 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65210b9c-f3b3-48a2-b187-644c8ec77813 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:56.976968 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58701d23-76a2-44e8-842a-4e405e1ba8d2 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:56.978676 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-58701d23-76a2-44e8-842a-4e405e1ba8d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:56.990357 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1046/2120] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:04:56 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:04:56.994483 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65210b9c-f3b3-48a2-b187-644c8ec77813 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:57.001513 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65210b9c-f3b3-48a2-b187-644c8ec77813 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:57.006330 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50159359-3c15-4c09-8fcd-f85310806d5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:57.007291 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50159359-3c15-4c09-8fcd-f85310806d5b admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:57.007574 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50159359-3c15-4c09-8fcd-f85310806d5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:57.007853 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50159359-3c15-4c09-8fcd-f85310806d5b admin admin] PATH_INFO: `/v3/projects/56996edc64864d55a0689264b135eff9/users/7a29c02f5c8d4da1999a19b38ddf4991/roles/b43d5a8247e244869c2c70f986f42e8a` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:57.024656 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50159359-3c15-4c09-8fcd-f85310806d5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=56996edc64864d55a0689264b135eff9, user_id=7a29c02f5c8d4da1999a19b38ddf4991, role_id=b43d5a8247e244869c2c70f986f42e8a)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:57.028091 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50159359-3c15-4c09-8fcd-f85310806d5b admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:57.038283 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58701d23-76a2-44e8-842a-4e405e1ba8d2 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:57.041967 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd18115-59ec-4e8a-b7ff-e50cbabce77e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:57.047739 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd18115-59ec-4e8a-b7ff-e50cbabce77e service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:57.047739 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd18115-59ec-4e8a-b7ff-e50cbabce77e service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:57.047739 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd18115-59ec-4e8a-b7ff-e50cbabce77e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:57.052146 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd18115-59ec-4e8a-b7ff-e50cbabce77e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:57.053562 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd18115-59ec-4e8a-b7ff-e50cbabce77e service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:57.057476 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:04:57.063729 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1075/2121] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:04:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:04:57.099321 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1047/2122] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:04:56 2022] PUT /identity/v3/projects/56996edc64864d55a0689264b135eff9/users/7a29c02f5c8d4da1999a19b38ddf4991/roles/b43d5a8247e244869c2c70f986f42e8a => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:04:57.103288 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd18115-59ec-4e8a-b7ff-e50cbabce77e service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:57.112875 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd18115-59ec-4e8a-b7ff-e50cbabce77e service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:57.168428 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcf0b1e-0572-4220-867d-88341a36327f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:57.169093 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcf0b1e-0572-4220-867d-88341a36327f admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:57.169376 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcf0b1e-0572-4220-867d-88341a36327f admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:57.169652 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcf0b1e-0572-4220-867d-88341a36327f admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:57.170399 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcf0b1e-0572-4220-867d-88341a36327f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:57.172008 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcf0b1e-0572-4220-867d-88341a36327f admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:57.185810 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1076/2123] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:04:57 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:04:57.186730 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50159359-3c15-4c09-8fcd-f85310806d5b admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:57.198002 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50159359-3c15-4c09-8fcd-f85310806d5b admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:57.203035 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e0402a-1610-45d3-b8c3-1f2daed6d293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:57.203949 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e0402a-1610-45d3-b8c3-1f2daed6d293 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:57.204235 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e0402a-1610-45d3-b8c3-1f2daed6d293 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:57.204512 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e0402a-1610-45d3-b8c3-1f2daed6d293 admin admin] PATH_INFO: `/v3/projects/56996edc64864d55a0689264b135eff9/users/7a29c02f5c8d4da1999a19b38ddf4991/roles/b43d5a8247e244869c2c70f986f42e8a` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:57.207055 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e0402a-1610-45d3-b8c3-1f2daed6d293 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56996edc64864d55a0689264b135eff9, user_id=7a29c02f5c8d4da1999a19b38ddf4991, role_id=b43d5a8247e244869c2c70f986f42e8a)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:57.209724 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e0402a-1610-45d3-b8c3-1f2daed6d293 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:57.220311 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1048/2124] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:04:57 2022] PUT /identity/v3/projects/56996edc64864d55a0689264b135eff9/users/7a29c02f5c8d4da1999a19b38ddf4991/roles/b43d5a8247e244869c2c70f986f42e8a => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:04:57.230271 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecc290a-808c-4da6-b143-56d45447a553 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:57.230562 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecc290a-808c-4da6-b143-56d45447a553 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:57.230836 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecc290a-808c-4da6-b143-56d45447a553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:57.257055 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-8ecc290a-808c-4da6-b143-56d45447a553 None None] MFA Rules not processed for user `7a29c02f5c8d4da1999a19b38ddf4991`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:04:57.265756 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8ecc290a-808c-4da6-b143-56d45447a553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:57.326372 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1077/2125] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:04:57 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:04:57.337032 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e0402a-1610-45d3-b8c3-1f2daed6d293 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:04:57.338768 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-61e0402a-1610-45d3-b8c3-1f2daed6d293 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:04:57.396811 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e0402a-1610-45d3-b8c3-1f2daed6d293 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:04:57.400725 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b97e6ed-f0ee-42ba-84ba-a78aafdb52bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6cc95a563b941ac92ab283aa0d5a5d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:04:57.401366 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b97e6ed-f0ee-42ba-84ba-a78aafdb52bf service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:04:57.401645 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b97e6ed-f0ee-42ba-84ba-a78aafdb52bf service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:04:57.401915 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b97e6ed-f0ee-42ba-84ba-a78aafdb52bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:04:57.461923 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b97e6ed-f0ee-42ba-84ba-a78aafdb52bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:04:57.463414 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b97e6ed-f0ee-42ba-84ba-a78aafdb52bf service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:04:57.468213 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:04:57.474679 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1049/2126] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:04:57 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:05:01.066724 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecc290a-808c-4da6-b143-56d45447a553 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:05:01.170427 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecc290a-808c-4da6-b143-56d45447a553 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:05:01.175681 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fd13d5-e646-49e1-8312-a9bae08e0621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:05:01.176692 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fd13d5-e646-49e1-8312-a9bae08e0621 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:01.177114 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fd13d5-e646-49e1-8312-a9bae08e0621 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:01.177396 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fd13d5-e646-49e1-8312-a9bae08e0621 admin admin] PATH_INFO: `/v3/users/e594b4b1ce424b6cbf571655a963cfc8` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:01.178926 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fd13d5-e646-49e1-8312-a9bae08e0621 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e594b4b1ce424b6cbf571655a963cfc8)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:05:01.182116 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fd13d5-e646-49e1-8312-a9bae08e0621 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:05:01.210340 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:05:01.432669 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e7fd13d5-e646-49e1-8312-a9bae08e0621 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e594b4b1ce424b6cbf571655a963cfc8'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:01.441770 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e7fd13d5-e646-49e1-8312-a9bae08e0621 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e594b4b1ce424b6cbf571655a963cfc8'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:01.550139 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e7fd13d5-e646-49e1-8312-a9bae08e0621 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e594b4b1ce424b6cbf571655a963cfc8'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:01.551329 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e7fd13d5-e646-49e1-8312-a9bae08e0621 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e594b4b1ce424b6cbf571655a963cfc8'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:01.565006 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e7fd13d5-e646-49e1-8312-a9bae08e0621 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e594b4b1ce424b6cbf571655a963cfc8'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:01.576680 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1078/2127] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:05:01 2022] DELETE /identity/v3/users/e594b4b1ce424b6cbf571655a963cfc8 => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:05:01.579965 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a71cdf-64fc-43a6-bf0d-8aff4848c7ba None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:01.580424 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a71cdf-64fc-43a6-bf0d-8aff4848c7ba None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:01.580535 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a71cdf-64fc-43a6-bf0d-8aff4848c7ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:01.613689 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-f1a71cdf-64fc-43a6-bf0d-8aff4848c7ba None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:05:01.615072 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f1a71cdf-64fc-43a6-bf0d-8aff4848c7ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:05:01.684154 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fd13d5-e646-49e1-8312-a9bae08e0621 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:05:01.685329 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e7fd13d5-e646-49e1-8312-a9bae08e0621 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:05:01.711743 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1050/2128] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:05:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:05:01.725787 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a71cdf-64fc-43a6-bf0d-8aff4848c7ba None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:05:01.728827 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f1a71cdf-64fc-43a6-bf0d-8aff4848c7ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:05:01.791121 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fd13d5-e646-49e1-8312-a9bae08e0621 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:05:01.797929 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1258381f-5bc3-4a7e-9d08-5dbc929f9f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:05:01.800942 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a71cdf-64fc-43a6-bf0d-8aff4848c7ba None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:05:01.801062 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1258381f-5bc3-4a7e-9d08-5dbc929f9f18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:01.801062 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1258381f-5bc3-4a7e-9d08-5dbc929f9f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:01.801062 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1258381f-5bc3-4a7e-9d08-5dbc929f9f18 admin admin] PATH_INFO: `/v3/users/785ada42968a40a88abbbdb69de55225` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:01.802039 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1258381f-5bc3-4a7e-9d08-5dbc929f9f18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=785ada42968a40a88abbbdb69de55225)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:05:01.802980 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbcf07d-ddf4-4389-ad8d-0c2b46a0018e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:05:01.803710 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbcf07d-ddf4-4389-ad8d-0c2b46a0018e service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:01.804117 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbcf07d-ddf4-4389-ad8d-0c2b46a0018e service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:01.804487 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbcf07d-ddf4-4389-ad8d-0c2b46a0018e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:01.804619 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1258381f-5bc3-4a7e-9d08-5dbc929f9f18 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:05:01.816658 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbcf07d-ddf4-4389-ad8d-0c2b46a0018e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:05:01.818215 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbcf07d-ddf4-4389-ad8d-0c2b46a0018e service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:05:01.831259 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1051/2129] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:05:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:05:02.085205 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-1258381f-5bc3-4a7e-9d08-5dbc929f9f18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '785ada42968a40a88abbbdb69de55225'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:02.093575 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-1258381f-5bc3-4a7e-9d08-5dbc929f9f18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '785ada42968a40a88abbbdb69de55225'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:02.191658 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-1258381f-5bc3-4a7e-9d08-5dbc929f9f18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '785ada42968a40a88abbbdb69de55225'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:02.192405 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-1258381f-5bc3-4a7e-9d08-5dbc929f9f18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '785ada42968a40a88abbbdb69de55225'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:02.230571 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-1258381f-5bc3-4a7e-9d08-5dbc929f9f18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '785ada42968a40a88abbbdb69de55225'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:02.238029 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1079/2130] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:05:01 2022] DELETE /identity/v3/users/785ada42968a40a88abbbdb69de55225 => generated 0 bytes in 555 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 10 13:05:02.242407 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe0478a-4741-4db0-b44c-d41f8dfd199b None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:02.242647 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe0478a-4741-4db0-b44c-d41f8dfd199b None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:02.242935 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe0478a-4741-4db0-b44c-d41f8dfd199b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:02.270917 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-4fe0478a-4741-4db0-b44c-d41f8dfd199b None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:05:02.280529 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-4fe0478a-4741-4db0-b44c-d41f8dfd199b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:05:02.364968 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1052/2131] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:05:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:05:02.400461 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1258381f-5bc3-4a7e-9d08-5dbc929f9f18 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:05:02.403928 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1258381f-5bc3-4a7e-9d08-5dbc929f9f18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:05:02.413108 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe0478a-4741-4db0-b44c-d41f8dfd199b None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:05:02.416929 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-4fe0478a-4741-4db0-b44c-d41f8dfd199b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:05:02.440459 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1258381f-5bc3-4a7e-9d08-5dbc929f9f18 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:05:02.464633 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735c1c4d-e866-4f32-8494-0b1ba028fcee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:05:02.467722 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735c1c4d-e866-4f32-8494-0b1ba028fcee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:02.469245 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735c1c4d-e866-4f32-8494-0b1ba028fcee admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:02.469734 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735c1c4d-e866-4f32-8494-0b1ba028fcee admin admin] PATH_INFO: `/v3/projects/bae19a1e72e34b5f9527462fe749b39e` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:02.472858 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735c1c4d-e866-4f32-8494-0b1ba028fcee admin admin] RBAC: Authorizing `identity:delete_project(project_id=bae19a1e72e34b5f9527462fe749b39e)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:05:02.479546 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735c1c4d-e866-4f32-8494-0b1ba028fcee admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:05:02.507591 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:05:02.534963 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:05:02.557708 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe0478a-4741-4db0-b44c-d41f8dfd199b None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:05:02.562684 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2862f47e-ff25-4c99-888f-5976912514a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:05:02.566077 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2862f47e-ff25-4c99-888f-5976912514a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:02.566077 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2862f47e-ff25-4c99-888f-5976912514a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:02.566077 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2862f47e-ff25-4c99-888f-5976912514a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:02.576534 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2862f47e-ff25-4c99-888f-5976912514a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:05:02.578345 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2862f47e-ff25-4c99-888f-5976912514a2 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:05:02.589881 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1053/2132] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:05:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:05:02.682973 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-735c1c4d-e866-4f32-8494-0b1ba028fcee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bae19a1e72e34b5f9527462fe749b39e', 'request_id': 'req-735c1c4d-e866-4f32-8494-0b1ba028fcee'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:02.695437 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-735c1c4d-e866-4f32-8494-0b1ba028fcee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bae19a1e72e34b5f9527462fe749b39e', 'request_id': 'req-735c1c4d-e866-4f32-8494-0b1ba028fcee'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:02.703348 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-735c1c4d-e866-4f32-8494-0b1ba028fcee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bae19a1e72e34b5f9527462fe749b39e', 'request_id': 'req-735c1c4d-e866-4f32-8494-0b1ba028fcee'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:02.774173 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-735c1c4d-e866-4f32-8494-0b1ba028fcee admin admin] The token cache is being invalidate because project bae19a1e72e34b5f9527462fe749b39e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:05:02.774476 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-735c1c4d-e866-4f32-8494-0b1ba028fcee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:02.777017 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-735c1c4d-e866-4f32-8494-0b1ba028fcee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:02.778857 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1080/2133] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:05:02 2022] DELETE /identity/v3/projects/bae19a1e72e34b5f9527462fe749b39e => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:05:02.784670 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d70c22-66ba-484d-a1bb-47ba3b7d2954 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:02.785006 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d70c22-66ba-484d-a1bb-47ba3b7d2954 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:02.785288 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d70c22-66ba-484d-a1bb-47ba3b7d2954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:02.832406 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-a4d70c22-66ba-484d-a1bb-47ba3b7d2954 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:05:02.834137 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a4d70c22-66ba-484d-a1bb-47ba3b7d2954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:05:02.916781 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1054/2134] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:05:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:05:02.918124 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b36cdf6-dda1-4442-8666-e38a0ebc1fc5 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:02.918437 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b36cdf6-dda1-4442-8666-e38a0ebc1fc5 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:02.918717 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b36cdf6-dda1-4442-8666-e38a0ebc1fc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:02.948850 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-3b36cdf6-dda1-4442-8666-e38a0ebc1fc5 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:05:02.948850 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3b36cdf6-dda1-4442-8666-e38a0ebc1fc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:05:02.975136 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d70c22-66ba-484d-a1bb-47ba3b7d2954 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:05:02.986871 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1081/2135] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:05:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 10 13:05:02.989517 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d70c22-66ba-484d-a1bb-47ba3b7d2954 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:05:03.013343 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4907e83-8093-408b-abc9-4d573090ae3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:05:03.016126 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4907e83-8093-408b-abc9-4d573090ae3b admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:03.019495 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4907e83-8093-408b-abc9-4d573090ae3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:03.019854 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4907e83-8093-408b-abc9-4d573090ae3b admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:03.020680 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4907e83-8093-408b-abc9-4d573090ae3b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 13:05:03.020997 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4907e83-8093-408b-abc9-4d573090ae3b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:05:03.022910 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4907e83-8093-408b-abc9-4d573090ae3b admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:05:03.060324 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b36cdf6-dda1-4442-8666-e38a0ebc1fc5 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:05:03.073838 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1055/2136] 10.222.0.27 () {58 vars in 1152 bytes} [Fri Jun 10 13:05:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 13:05:03.074245 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b36cdf6-dda1-4442-8666-e38a0ebc1fc5 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:05:03.082831 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd91cc81-06e1-4746-8ede-7d42748c9233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:05:03.084065 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd91cc81-06e1-4746-8ede-7d42748c9233 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:03.084551 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd91cc81-06e1-4746-8ede-7d42748c9233 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:03.085247 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd91cc81-06e1-4746-8ede-7d42748c9233 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:03.087084 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd91cc81-06e1-4746-8ede-7d42748c9233 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:05:03.091258 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd91cc81-06e1-4746-8ede-7d42748c9233 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:05:03.252259 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1082/2137] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:05:03 2022] POST /identity/v3/projects => generated 375 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:05:03.256090 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4907e83-8093-408b-abc9-4d573090ae3b admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:05:03.272597 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4907e83-8093-408b-abc9-4d573090ae3b admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:05:03.409156 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfede827-21d0-43b4-b476-807da499cd55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:05:03.410163 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfede827-21d0-43b4-b476-807da499cd55 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:03.410625 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfede827-21d0-43b4-b476-807da499cd55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:03.411391 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfede827-21d0-43b4-b476-807da499cd55 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:03.412579 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfede827-21d0-43b4-b476-807da499cd55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:05:03.414990 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfede827-21d0-43b4-b476-807da499cd55 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:05:03.433782 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd91cc81-06e1-4746-8ede-7d42748c9233 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:05:03.436208 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-fd91cc81-06e1-4746-8ede-7d42748c9233 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:05:03.460681 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd91cc81-06e1-4746-8ede-7d42748c9233 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:05:03.469990 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c330da77-159f-4416-a101-b91509971b91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:05:03.471659 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c330da77-159f-4416-a101-b91509971b91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:03.472396 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c330da77-159f-4416-a101-b91509971b91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:03.472910 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c330da77-159f-4416-a101-b91509971b91 admin admin] PATH_INFO: `/v3/projects/11f9adc51a60485d8c2a745d92782d76` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:03.475772 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c330da77-159f-4416-a101-b91509971b91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=11f9adc51a60485d8c2a745d92782d76)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:05:03.478732 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c330da77-159f-4416-a101-b91509971b91 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:05:03.514425 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1056/2138] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:05:03 2022] POST /identity/v3/users => generated 288 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:05:03.518543 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfede827-21d0-43b4-b476-807da499cd55 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:05:03.531383 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfede827-21d0-43b4-b476-807da499cd55 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:05:03.535891 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f717ae32-a080-4c14-8da6-397afe4197dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:05:03.536568 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f717ae32-a080-4c14-8da6-397afe4197dc admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:03.536854 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f717ae32-a080-4c14-8da6-397afe4197dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:03.537525 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f717ae32-a080-4c14-8da6-397afe4197dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:03.539200 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f717ae32-a080-4c14-8da6-397afe4197dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:05:03.541332 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f717ae32-a080-4c14-8da6-397afe4197dc admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:05:03.552950 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1057/2139] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:05:03 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 10 13:05:03.562372 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f717ae32-a080-4c14-8da6-397afe4197dc admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:05:03.578413 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f717ae32-a080-4c14-8da6-397afe4197dc admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:05:03.583939 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9563cdc-4586-4241-be06-0536e83d3da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:05:03.585300 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9563cdc-4586-4241-be06-0536e83d3da1 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:03.585586 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9563cdc-4586-4241-be06-0536e83d3da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:03.585863 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9563cdc-4586-4241-be06-0536e83d3da1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:03.587074 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9563cdc-4586-4241-be06-0536e83d3da1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:05:03.590114 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9563cdc-4586-4241-be06-0536e83d3da1 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:05:03.608452 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1058/2140] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:05:03 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 10 13:05:03.609924 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359766b3-6b48-4f17-81a0-7950adaee7f7 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:03.610273 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359766b3-6b48-4f17-81a0-7950adaee7f7 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:03.610560 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359766b3-6b48-4f17-81a0-7950adaee7f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:03.628543 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c330da77-159f-4416-a101-b91509971b91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11f9adc51a60485d8c2a745d92782d76', 'request_id': 'req-c330da77-159f-4416-a101-b91509971b91'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:03.634037 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c330da77-159f-4416-a101-b91509971b91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11f9adc51a60485d8c2a745d92782d76', 'request_id': 'req-c330da77-159f-4416-a101-b91509971b91'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:03.641522 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c330da77-159f-4416-a101-b91509971b91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11f9adc51a60485d8c2a745d92782d76', 'request_id': 'req-c330da77-159f-4416-a101-b91509971b91'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:03.649248 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-359766b3-6b48-4f17-81a0-7950adaee7f7 None None] MFA Rules not processed for user `a72b5e893615479ea3fe1838d07496ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:05:03.650794 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-359766b3-6b48-4f17-81a0-7950adaee7f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:05:03.687102 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:05:03.709908 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c330da77-159f-4416-a101-b91509971b91 admin admin] The token cache is being invalidate because project 11f9adc51a60485d8c2a745d92782d76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:05:03.710492 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c330da77-159f-4416-a101-b91509971b91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:03.711323 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c330da77-159f-4416-a101-b91509971b91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:03.713391 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1083/2141] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:05:03 2022] DELETE /identity/v3/projects/11f9adc51a60485d8c2a745d92782d76 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:05:03.714418 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c330da77-159f-4416-a101-b91509971b91 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:05:03.715998 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c330da77-159f-4416-a101-b91509971b91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:05:03.728488 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1059/2142] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:05:03 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:05:03.728488 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359766b3-6b48-4f17-81a0-7950adaee7f7 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:05:03.795523 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:05:03.798239 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359766b3-6b48-4f17-81a0-7950adaee7f7 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:05:03.801033 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c330da77-159f-4416-a101-b91509971b91 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:05:03.802904 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35413419-c220-42d9-90a2-ab665e5318d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:05:03.803786 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35413419-c220-42d9-90a2-ab665e5318d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:03.804107 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35413419-c220-42d9-90a2-ab665e5318d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:03.804439 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35413419-c220-42d9-90a2-ab665e5318d1 admin admin] PATH_INFO: `/v3/users/7a29c02f5c8d4da1999a19b38ddf4991` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:03.805809 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35413419-c220-42d9-90a2-ab665e5318d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a29c02f5c8d4da1999a19b38ddf4991)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:05:03.807460 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792313bb-7305-49c2-8c1c-cfaab72e85d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:05:03.808620 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792313bb-7305-49c2-8c1c-cfaab72e85d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:03.808827 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35413419-c220-42d9-90a2-ab665e5318d1 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:05:03.809203 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792313bb-7305-49c2-8c1c-cfaab72e85d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:03.809642 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792313bb-7305-49c2-8c1c-cfaab72e85d8 admin admin] PATH_INFO: `/v3/projects/ea02b87433cf43f6824bfcbd8245142a/users/ea3ad28e5b9d48c9acbf4ea3d554fb1f/roles/43106c664e1b40de984c0c62701149c8` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:03.830027 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792313bb-7305-49c2-8c1c-cfaab72e85d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea02b87433cf43f6824bfcbd8245142a, user_id=ea3ad28e5b9d48c9acbf4ea3d554fb1f, role_id=43106c664e1b40de984c0c62701149c8)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:05:03.832817 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792313bb-7305-49c2-8c1c-cfaab72e85d8 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:05:03.916657 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1084/2143] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:05:03 2022] PUT /identity/v3/projects/ea02b87433cf43f6824bfcbd8245142a/users/ea3ad28e5b9d48c9acbf4ea3d554fb1f/roles/43106c664e1b40de984c0c62701149c8 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:05:03.917465 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792313bb-7305-49c2-8c1c-cfaab72e85d8 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:05:03.919064 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-792313bb-7305-49c2-8c1c-cfaab72e85d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:05:03.998858 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792313bb-7305-49c2-8c1c-cfaab72e85d8 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:05:04.003897 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40318f49-6f76-4733-afb4-006d8e7af02a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:05:04.004359 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40318f49-6f76-4733-afb4-006d8e7af02a admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:04.004645 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40318f49-6f76-4733-afb4-006d8e7af02a admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:04.004931 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40318f49-6f76-4733-afb4-006d8e7af02a admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:04.005599 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40318f49-6f76-4733-afb4-006d8e7af02a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:05:04.007514 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40318f49-6f76-4733-afb4-006d8e7af02a admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:05:04.061126 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-35413419-c220-42d9-90a2-ab665e5318d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a29c02f5c8d4da1999a19b38ddf4991'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:04.067988 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-35413419-c220-42d9-90a2-ab665e5318d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a29c02f5c8d4da1999a19b38ddf4991'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:04.103963 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1085/2144] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:05:03 2022] POST /identity/v3/projects => generated 371 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:05:04.103963 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329eaf9b-5543-43a3-b69b-b554a052ae5e None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:04.103963 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329eaf9b-5543-43a3-b69b-b554a052ae5e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:04.103963 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329eaf9b-5543-43a3-b69b-b554a052ae5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:04.130759 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-329eaf9b-5543-43a3-b69b-b554a052ae5e None None] MFA Rules not processed for user `ea3ad28e5b9d48c9acbf4ea3d554fb1f`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:05:04.132136 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-329eaf9b-5543-43a3-b69b-b554a052ae5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:05:04.171360 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-35413419-c220-42d9-90a2-ab665e5318d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a29c02f5c8d4da1999a19b38ddf4991'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:04.172245 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-35413419-c220-42d9-90a2-ab665e5318d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a29c02f5c8d4da1999a19b38ddf4991'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:04.203666 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-35413419-c220-42d9-90a2-ab665e5318d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a29c02f5c8d4da1999a19b38ddf4991'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:04.213318 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1060/2145] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:05:03 2022] DELETE /identity/v3/users/7a29c02f5c8d4da1999a19b38ddf4991 => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:05:04.217308 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35413419-c220-42d9-90a2-ab665e5318d1 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:05:04.221073 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1086/2146] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:05:04 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:05:04.221492 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-35413419-c220-42d9-90a2-ab665e5318d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:05:04.228437 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329eaf9b-5543-43a3-b69b-b554a052ae5e None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:05:04.343974 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:05:04.366850 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329eaf9b-5543-43a3-b69b-b554a052ae5e None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:05:04.375412 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35413419-c220-42d9-90a2-ab665e5318d1 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:05:04.380948 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99d20ed-3c9a-4698-b5f3-10663d3d213d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:05:04.382225 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99d20ed-3c9a-4698-b5f3-10663d3d213d admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:04.382371 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99d20ed-3c9a-4698-b5f3-10663d3d213d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:04.382788 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99d20ed-3c9a-4698-b5f3-10663d3d213d admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:04.383705 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99d20ed-3c9a-4698-b5f3-10663d3d213d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:05:04.384686 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689c9d86-a011-471e-b1e3-add4ab3f0ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:05:04.385571 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689c9d86-a011-471e-b1e3-add4ab3f0ce3 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:04.386015 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689c9d86-a011-471e-b1e3-add4ab3f0ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:04.386438 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689c9d86-a011-471e-b1e3-add4ab3f0ce3 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:04.387454 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689c9d86-a011-471e-b1e3-add4ab3f0ce3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:05:04.396493 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689c9d86-a011-471e-b1e3-add4ab3f0ce3 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:05:04.399142 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99d20ed-3c9a-4698-b5f3-10663d3d213d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:05:04.504033 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1061/2147] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:05:04 2022] POST /identity/v3/users => generated 292 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:05:04.508456 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689c9d86-a011-471e-b1e3-add4ab3f0ce3 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:05:04.509444 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1087/2148] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:05:04 2022] POST /identity/v3/projects => generated 375 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:05:04.512611 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99d20ed-3c9a-4698-b5f3-10663d3d213d admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:05:04.514612 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b99d20ed-3c9a-4698-b5f3-10663d3d213d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:05:04.534683 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689c9d86-a011-471e-b1e3-add4ab3f0ce3 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:05:04.655970 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc264853-0dc0-47bc-b3ab-41e6edeb3d98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:05:04.655970 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc264853-0dc0-47bc-b3ab-41e6edeb3d98 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:04.655970 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc264853-0dc0-47bc-b3ab-41e6edeb3d98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:04.655970 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc264853-0dc0-47bc-b3ab-41e6edeb3d98 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:04.656796 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc264853-0dc0-47bc-b3ab-41e6edeb3d98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:05:04.656796 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc264853-0dc0-47bc-b3ab-41e6edeb3d98 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:05:04.666386 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1062/2149] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:05:04 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:05:04.671344 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc264853-0dc0-47bc-b3ab-41e6edeb3d98 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:05:04.676270 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:05:04.687453 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc264853-0dc0-47bc-b3ab-41e6edeb3d98 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:05:04.693129 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99d20ed-3c9a-4698-b5f3-10663d3d213d admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:05:04.693684 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cadafe-2173-4903-942a-362589ccd450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:05:04.694807 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cadafe-2173-4903-942a-362589ccd450 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:04.695057 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cadafe-2173-4903-942a-362589ccd450 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:04.695746 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cadafe-2173-4903-942a-362589ccd450 admin admin] PATH_INFO: `/v3/projects/9f2d9d35e5e5436a9a3b84313d20fb67/users/869c7d82fde44c4083eeb51dc91b2b61/roles/b43d5a8247e244869c2c70f986f42e8a` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:04.707617 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90b6ecb-24c2-49bd-b187-0f197a8ecbda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:05:04.709125 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90b6ecb-24c2-49bd-b187-0f197a8ecbda admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:04.709640 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90b6ecb-24c2-49bd-b187-0f197a8ecbda admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:04.709927 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90b6ecb-24c2-49bd-b187-0f197a8ecbda admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:04.712847 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90b6ecb-24c2-49bd-b187-0f197a8ecbda admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:05:04.715494 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cadafe-2173-4903-942a-362589ccd450 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f2d9d35e5e5436a9a3b84313d20fb67, user_id=869c7d82fde44c4083eeb51dc91b2b61, role_id=b43d5a8247e244869c2c70f986f42e8a)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:05:04.715665 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90b6ecb-24c2-49bd-b187-0f197a8ecbda admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:05:04.718589 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cadafe-2173-4903-942a-362589ccd450 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:05:04.828093 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1088/2150] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:05:04 2022] POST /identity/v3/users => generated 288 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:05:04.828093 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90b6ecb-24c2-49bd-b187-0f197a8ecbda admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:05:04.833435 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e90b6ecb-24c2-49bd-b187-0f197a8ecbda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:05:04.838843 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1063/2151] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:05:04 2022] PUT /identity/v3/projects/9f2d9d35e5e5436a9a3b84313d20fb67/users/869c7d82fde44c4083eeb51dc91b2b61/roles/b43d5a8247e244869c2c70f986f42e8a => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:05:04.839816 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cadafe-2173-4903-942a-362589ccd450 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:05:04.848241 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90b6ecb-24c2-49bd-b187-0f197a8ecbda admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:05:04.933297 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90828e55-a593-4848-ac2b-fb39148cdb95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:05:04.934185 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90828e55-a593-4848-ac2b-fb39148cdb95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:04.934470 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90828e55-a593-4848-ac2b-fb39148cdb95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:04.934747 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90828e55-a593-4848-ac2b-fb39148cdb95 admin admin] PATH_INFO: `/v3/projects/56996edc64864d55a0689264b135eff9` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:04.936115 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90828e55-a593-4848-ac2b-fb39148cdb95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=56996edc64864d55a0689264b135eff9)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:05:04.936254 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cadafe-2173-4903-942a-362589ccd450 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:05:04.938021 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90828e55-a593-4848-ac2b-fb39148cdb95 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:05:04.941661 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14300ff0-5527-42fc-b8f9-10ef3328cb6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:05:04.942409 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14300ff0-5527-42fc-b8f9-10ef3328cb6b admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:04.942755 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14300ff0-5527-42fc-b8f9-10ef3328cb6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:04.943095 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14300ff0-5527-42fc-b8f9-10ef3328cb6b admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:04.944004 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14300ff0-5527-42fc-b8f9-10ef3328cb6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:05:04.945625 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14300ff0-5527-42fc-b8f9-10ef3328cb6b admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:05:04.956825 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1064/2152] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:05:04 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:05:04.957879 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14300ff0-5527-42fc-b8f9-10ef3328cb6b admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:05:04.972284 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14300ff0-5527-42fc-b8f9-10ef3328cb6b admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:05:04.977402 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a4bb93-81cd-4c46-bae5-05b3cd1d5e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:05:04.978832 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a4bb93-81cd-4c46-bae5-05b3cd1d5e51 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:04.979184 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a4bb93-81cd-4c46-bae5-05b3cd1d5e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:04.979532 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a4bb93-81cd-4c46-bae5-05b3cd1d5e51 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:04.980498 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a4bb93-81cd-4c46-bae5-05b3cd1d5e51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:05:04.982244 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a4bb93-81cd-4c46-bae5-05b3cd1d5e51 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:05:04.993863 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1065/2153] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:05:04 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:05:04.997598 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a4bb93-81cd-4c46-bae5-05b3cd1d5e51 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:05:05.011942 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a4bb93-81cd-4c46-bae5-05b3cd1d5e51 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:05:05.019117 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78951b5-5631-4104-83f7-61b1b17ec5dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:05:05.020120 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78951b5-5631-4104-83f7-61b1b17ec5dc admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:05.020556 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78951b5-5631-4104-83f7-61b1b17ec5dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:05.020859 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78951b5-5631-4104-83f7-61b1b17ec5dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:05.022918 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78951b5-5631-4104-83f7-61b1b17ec5dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:05:05.025726 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78951b5-5631-4104-83f7-61b1b17ec5dc admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:05:05.050730 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1066/2154] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:05:04 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:05:05.050730 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78951b5-5631-4104-83f7-61b1b17ec5dc admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:05:05.062806 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78951b5-5631-4104-83f7-61b1b17ec5dc admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:05:05.068730 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6e8390-31fe-4cd3-b07d-55ddeaf69c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:05:05.069624 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6e8390-31fe-4cd3-b07d-55ddeaf69c9d admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:05.069918 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6e8390-31fe-4cd3-b07d-55ddeaf69c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:05.070196 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6e8390-31fe-4cd3-b07d-55ddeaf69c9d admin admin] PATH_INFO: `/v3/projects/9f2d9d35e5e5436a9a3b84313d20fb67/users/869c7d82fde44c4083eeb51dc91b2b61/roles/b43d5a8247e244869c2c70f986f42e8a` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:05.072828 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6e8390-31fe-4cd3-b07d-55ddeaf69c9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f2d9d35e5e5436a9a3b84313d20fb67, user_id=869c7d82fde44c4083eeb51dc91b2b61, role_id=b43d5a8247e244869c2c70f986f42e8a)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:05:05.076231 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6e8390-31fe-4cd3-b07d-55ddeaf69c9d admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:05:05.091945 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1067/2155] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:05:05 2022] PUT /identity/v3/projects/9f2d9d35e5e5436a9a3b84313d20fb67/users/869c7d82fde44c4083eeb51dc91b2b61/roles/b43d5a8247e244869c2c70f986f42e8a => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:05:05.093060 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6e8390-31fe-4cd3-b07d-55ddeaf69c9d admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:05:05.100848 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-90828e55-a593-4848-ac2b-fb39148cdb95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56996edc64864d55a0689264b135eff9', 'request_id': 'req-90828e55-a593-4848-ac2b-fb39148cdb95'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:05.104998 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-90828e55-a593-4848-ac2b-fb39148cdb95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56996edc64864d55a0689264b135eff9', 'request_id': 'req-90828e55-a593-4848-ac2b-fb39148cdb95'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:05.112595 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-90828e55-a593-4848-ac2b-fb39148cdb95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56996edc64864d55a0689264b135eff9', 'request_id': 'req-90828e55-a593-4848-ac2b-fb39148cdb95'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:05.171510 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:05:05.173615 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-90828e55-a593-4848-ac2b-fb39148cdb95 admin admin] The token cache is being invalidate because project 56996edc64864d55a0689264b135eff9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:05:05.174111 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-90828e55-a593-4848-ac2b-fb39148cdb95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:05.175431 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-90828e55-a593-4848-ac2b-fb39148cdb95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:05.179997 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1089/2156] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:05:04 2022] DELETE /identity/v3/projects/56996edc64864d55a0689264b135eff9 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:05:05.184286 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a53e0bd-5b68-43b2-b65f-c125970c8fa3 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:05.184847 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a53e0bd-5b68-43b2-b65f-c125970c8fa3 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:05.185372 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a53e0bd-5b68-43b2-b65f-c125970c8fa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:05.188268 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6e8390-31fe-4cd3-b07d-55ddeaf69c9d admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:05:05.197133 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d48cd3f-0e37-4cf6-8409-6d6214bc23de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:05:05.198075 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d48cd3f-0e37-4cf6-8409-6d6214bc23de admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:05.198380 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d48cd3f-0e37-4cf6-8409-6d6214bc23de admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:05.198636 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d48cd3f-0e37-4cf6-8409-6d6214bc23de admin admin] PATH_INFO: `/v3/projects/d17d6695ebdd4302be23304b37a5646a/users/9acccd4e69014cbeb566f94a8a579fec/roles/43106c664e1b40de984c0c62701149c8` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:05.208095 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-4a53e0bd-5b68-43b2-b65f-c125970c8fa3 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:05:05.209588 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-4a53e0bd-5b68-43b2-b65f-c125970c8fa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:05:05.219263 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1090/2157] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:05:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:05:05.226716 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153cb58d-868d-45e3-877d-e549da8eedf0 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:05.227235 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153cb58d-868d-45e3-877d-e549da8eedf0 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:05.227691 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d48cd3f-0e37-4cf6-8409-6d6214bc23de admin admin] RBAC: Authorizing `identity:create_grant(project_id=d17d6695ebdd4302be23304b37a5646a, user_id=9acccd4e69014cbeb566f94a8a579fec, role_id=43106c664e1b40de984c0c62701149c8)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:05:05.227985 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153cb58d-868d-45e3-877d-e549da8eedf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:05.230579 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d48cd3f-0e37-4cf6-8409-6d6214bc23de admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:05:05.248742 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-153cb58d-868d-45e3-877d-e549da8eedf0 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:05:05.248742 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-153cb58d-868d-45e3-877d-e549da8eedf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:05:05.258970 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1091/2158] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:05:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:05:05.263752 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153cb58d-868d-45e3-877d-e549da8eedf0 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:05:05.273196 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153cb58d-868d-45e3-877d-e549da8eedf0 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:05:05.278744 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7a8bd9-618b-4082-a6cc-6cfba7e32384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:05:05.279503 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7a8bd9-618b-4082-a6cc-6cfba7e32384 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:05.279886 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7a8bd9-618b-4082-a6cc-6cfba7e32384 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:05.280237 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7a8bd9-618b-4082-a6cc-6cfba7e32384 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:05.281045 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7a8bd9-618b-4082-a6cc-6cfba7e32384 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 13:05:05.281423 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7a8bd9-618b-4082-a6cc-6cfba7e32384 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:05:05.283120 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7a8bd9-618b-4082-a6cc-6cfba7e32384 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:05:05.294981 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1092/2159] 10.222.0.27 () {58 vars in 1152 bytes} [Fri Jun 10 13:05:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 13:05:05.299383 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7a8bd9-618b-4082-a6cc-6cfba7e32384 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:05:05.309611 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7a8bd9-618b-4082-a6cc-6cfba7e32384 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:05:05.314818 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32254238-a027-4da8-8eb1-99b89200e2dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:05:05.315547 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32254238-a027-4da8-8eb1-99b89200e2dc admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:05.316012 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32254238-a027-4da8-8eb1-99b89200e2dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:05.316317 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32254238-a027-4da8-8eb1-99b89200e2dc admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:05.317085 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32254238-a027-4da8-8eb1-99b89200e2dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:05:05.319394 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32254238-a027-4da8-8eb1-99b89200e2dc admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:05:05.354513 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1068/2160] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:05:05 2022] PUT /identity/v3/projects/d17d6695ebdd4302be23304b37a5646a/users/9acccd4e69014cbeb566f94a8a579fec/roles/43106c664e1b40de984c0c62701149c8 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:05:05.364288 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6475cbd-bfea-45ef-829c-1423f0f7e0fe None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:05.364994 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6475cbd-bfea-45ef-829c-1423f0f7e0fe None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:05.366014 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6475cbd-bfea-45ef-829c-1423f0f7e0fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:05.396297 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-f6475cbd-bfea-45ef-829c-1423f0f7e0fe None None] MFA Rules not processed for user `9acccd4e69014cbeb566f94a8a579fec`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:05:05.398051 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f6475cbd-bfea-45ef-829c-1423f0f7e0fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:05:05.436087 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1093/2161] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:05:05 2022] POST /identity/v3/projects => generated 385 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:05:05.437258 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32254238-a027-4da8-8eb1-99b89200e2dc admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:05:05.454199 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:05:05.486577 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1069/2162] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:05:05 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 10 13:05:05.497764 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6475cbd-bfea-45ef-829c-1423f0f7e0fe None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:05:05.497764 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f6475cbd-bfea-45ef-829c-1423f0f7e0fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:05:05.572257 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32254238-a027-4da8-8eb1-99b89200e2dc admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:05:05.578088 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fafb5c-2e20-4fbd-8a98-af70ce80f0cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:05:05.578791 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fafb5c-2e20-4fbd-8a98-af70ce80f0cf admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:05.579087 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fafb5c-2e20-4fbd-8a98-af70ce80f0cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:05.579371 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fafb5c-2e20-4fbd-8a98-af70ce80f0cf admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:05.580196 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fafb5c-2e20-4fbd-8a98-af70ce80f0cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:05:05.582260 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fafb5c-2e20-4fbd-8a98-af70ce80f0cf admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:05:05.617167 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6475cbd-bfea-45ef-829c-1423f0f7e0fe None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:05:05.622990 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aeaf52d-696c-4341-937f-7a2cf8c3d7f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:05:05.623732 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeaf52d-696c-4341-937f-7a2cf8c3d7f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:05.624079 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeaf52d-696c-4341-937f-7a2cf8c3d7f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:05.624361 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aeaf52d-696c-4341-937f-7a2cf8c3d7f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:05.625074 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aeaf52d-696c-4341-937f-7a2cf8c3d7f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:05:05.628873 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aeaf52d-696c-4341-937f-7a2cf8c3d7f3 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:05:05.731643 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1094/2163] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:05:05 2022] POST /identity/v3/users => generated 293 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 10 13:05:05.733931 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fafb5c-2e20-4fbd-8a98-af70ce80f0cf admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:05:05.746129 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fafb5c-2e20-4fbd-8a98-af70ce80f0cf admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:05:05.752348 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a15edb1-1bad-4f5e-986c-75842807d1ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:05:05.753074 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a15edb1-1bad-4f5e-986c-75842807d1ba admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:05.753358 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a15edb1-1bad-4f5e-986c-75842807d1ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:05.753686 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a15edb1-1bad-4f5e-986c-75842807d1ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:05.767056 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a15edb1-1bad-4f5e-986c-75842807d1ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:05:05.767056 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a15edb1-1bad-4f5e-986c-75842807d1ba admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:05:05.767518 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1070/2164] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:05:05 2022] POST /identity/v3/projects => generated 375 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:05:05.767987 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1095/2165] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:05:05 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:05:05.770757 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a15edb1-1bad-4f5e-986c-75842807d1ba admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:05:05.771034 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aeaf52d-696c-4341-937f-7a2cf8c3d7f3 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:05:05.781891 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a15edb1-1bad-4f5e-986c-75842807d1ba admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:05:05.849163 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aeaf52d-696c-4341-937f-7a2cf8c3d7f3 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:05:05.855334 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa4ea35-e153-4e83-8b55-0368f701014c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:05:05.856108 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa4ea35-e153-4e83-8b55-0368f701014c admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:05.856392 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa4ea35-e153-4e83-8b55-0368f701014c admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:05.858177 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa4ea35-e153-4e83-8b55-0368f701014c admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:05.859133 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa4ea35-e153-4e83-8b55-0368f701014c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:05:05.861124 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa4ea35-e153-4e83-8b55-0368f701014c admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:05:05.864846 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:05:05.892678 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29921c4-ef85-49f0-be7e-abdf3a777ea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:05:05.893521 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29921c4-ef85-49f0-be7e-abdf3a777ea5 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:05.893873 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29921c4-ef85-49f0-be7e-abdf3a777ea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:05.894219 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29921c4-ef85-49f0-be7e-abdf3a777ea5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:05.895396 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29921c4-ef85-49f0-be7e-abdf3a777ea5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:05:05.897064 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29921c4-ef85-49f0-be7e-abdf3a777ea5 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:05:05.916390 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1096/2166] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:05:05 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:05:05.919447 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29921c4-ef85-49f0-be7e-abdf3a777ea5 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:05:05.940077 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29921c4-ef85-49f0-be7e-abdf3a777ea5 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:05:05.942525 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c33b864-e7f7-4ae1-bfde-605d1c41aa6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:05:05.943506 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c33b864-e7f7-4ae1-bfde-605d1c41aa6a admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:05.943787 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c33b864-e7f7-4ae1-bfde-605d1c41aa6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:05.944266 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c33b864-e7f7-4ae1-bfde-605d1c41aa6a admin admin] PATH_INFO: `/v3/projects/3e56e08954de404a8ac6db8fdfcafe76/users/5da72979b3774c62a6963e0b641923bf/roles/43106c664e1b40de984c0c62701149c8` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:05.954998 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1071/2167] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:05:05 2022] POST /identity/v3/users => generated 294 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:05:05.963469 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c33b864-e7f7-4ae1-bfde-605d1c41aa6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e56e08954de404a8ac6db8fdfcafe76, user_id=5da72979b3774c62a6963e0b641923bf, role_id=43106c664e1b40de984c0c62701149c8)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:05:05.964907 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa4ea35-e153-4e83-8b55-0368f701014c admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:05:05.968933 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c33b864-e7f7-4ae1-bfde-605d1c41aa6a admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:05:05.977518 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa4ea35-e153-4e83-8b55-0368f701014c admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:05:05.984459 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658d400f-7c36-4318-8491-a895d6a84f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:05:05.984953 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658d400f-7c36-4318-8491-a895d6a84f0a admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:05.985349 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658d400f-7c36-4318-8491-a895d6a84f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:05.985575 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658d400f-7c36-4318-8491-a895d6a84f0a admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:05.986438 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658d400f-7c36-4318-8491-a895d6a84f0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:05:05.988705 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658d400f-7c36-4318-8491-a895d6a84f0a admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:05:06.000441 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1072/2168] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:05:05 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:05:06.004520 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658d400f-7c36-4318-8491-a895d6a84f0a admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:05:06.015455 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658d400f-7c36-4318-8491-a895d6a84f0a admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:05:06.020481 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4867434f-31e1-45d4-9bb6-8bd9e6deaa90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:05:06.021536 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4867434f-31e1-45d4-9bb6-8bd9e6deaa90 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:06.021714 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4867434f-31e1-45d4-9bb6-8bd9e6deaa90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:06.021976 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4867434f-31e1-45d4-9bb6-8bd9e6deaa90 admin admin] PATH_INFO: `/v3/projects/26d4bc05d9db455e8f72be806503d06c/users/eace14435db74925858cdce3fd70ca56/roles/b43d5a8247e244869c2c70f986f42e8a` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:06.057214 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4867434f-31e1-45d4-9bb6-8bd9e6deaa90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26d4bc05d9db455e8f72be806503d06c, user_id=eace14435db74925858cdce3fd70ca56, role_id=b43d5a8247e244869c2c70f986f42e8a)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:05:06.060929 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4867434f-31e1-45d4-9bb6-8bd9e6deaa90 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:05:06.081611 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1097/2169] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:05:05 2022] PUT /identity/v3/projects/3e56e08954de404a8ac6db8fdfcafe76/users/5da72979b3774c62a6963e0b641923bf/roles/43106c664e1b40de984c0c62701149c8 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:05:06.095515 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338b276e-92aa-49ad-9f5d-57bcf585daba None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:06.096070 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338b276e-92aa-49ad-9f5d-57bcf585daba None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:06.096390 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338b276e-92aa-49ad-9f5d-57bcf585daba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:06.141864 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-338b276e-92aa-49ad-9f5d-57bcf585daba None None] MFA Rules not processed for user `5da72979b3774c62a6963e0b641923bf`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:05:06.143295 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-338b276e-92aa-49ad-9f5d-57bcf585daba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:05:06.179877 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1073/2170] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:05:06 2022] PUT /identity/v3/projects/26d4bc05d9db455e8f72be806503d06c/users/eace14435db74925858cdce3fd70ca56/roles/b43d5a8247e244869c2c70f986f42e8a => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:05:06.181001 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4867434f-31e1-45d4-9bb6-8bd9e6deaa90 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:05:06.289952 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1098/2171] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:05:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:05:06.291432 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338b276e-92aa-49ad-9f5d-57bcf585daba None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:05:06.378457 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4867434f-31e1-45d4-9bb6-8bd9e6deaa90 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:05:06.384979 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8ccd7b-74a6-4729-afaf-98f762cfa236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:05:06.385890 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8ccd7b-74a6-4729-afaf-98f762cfa236 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:06.386732 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8ccd7b-74a6-4729-afaf-98f762cfa236 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:06.386732 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8ccd7b-74a6-4729-afaf-98f762cfa236 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:06.389350 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8ccd7b-74a6-4729-afaf-98f762cfa236 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:05:06.389473 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8ccd7b-74a6-4729-afaf-98f762cfa236 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:05:06.399446 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338b276e-92aa-49ad-9f5d-57bcf585daba None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:05:06.407176 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:05:06.411432 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a59210-aadb-4059-9abb-de8c6c857c28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:05:06.414959 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a59210-aadb-4059-9abb-de8c6c857c28 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:06.415087 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1074/2172] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:05:06 2022] GET /identity/v3/roles => generated 1420 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:05:06.416928 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a59210-aadb-4059-9abb-de8c6c857c28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:06.417374 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a59210-aadb-4059-9abb-de8c6c857c28 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:06.417836 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8ccd7b-74a6-4729-afaf-98f762cfa236 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:05:06.418350 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a59210-aadb-4059-9abb-de8c6c857c28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:05:06.420503 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a59210-aadb-4059-9abb-de8c6c857c28 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:05:06.430916 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8ccd7b-74a6-4729-afaf-98f762cfa236 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:05:06.437032 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd152c55-c2cb-47e7-9f55-a57ed226135a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:05:06.438327 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd152c55-c2cb-47e7-9f55-a57ed226135a admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:06.438611 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd152c55-c2cb-47e7-9f55-a57ed226135a admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:06.438889 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd152c55-c2cb-47e7-9f55-a57ed226135a admin admin] PATH_INFO: `/v3/projects/26d4bc05d9db455e8f72be806503d06c/users/eace14435db74925858cdce3fd70ca56/roles/b43d5a8247e244869c2c70f986f42e8a` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:06.441497 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd152c55-c2cb-47e7-9f55-a57ed226135a admin admin] RBAC: Authorizing `identity:create_grant(project_id=26d4bc05d9db455e8f72be806503d06c, user_id=eace14435db74925858cdce3fd70ca56, role_id=b43d5a8247e244869c2c70f986f42e8a)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:05:06.444360 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd152c55-c2cb-47e7-9f55-a57ed226135a admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:05:06.456154 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1075/2173] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:05:06 2022] PUT /identity/v3/projects/26d4bc05d9db455e8f72be806503d06c/users/eace14435db74925858cdce3fd70ca56/roles/b43d5a8247e244869c2c70f986f42e8a => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:05:06.467312 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf68593-e605-48df-bd45-505253c0d115 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:06.468051 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf68593-e605-48df-bd45-505253c0d115 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:06.468475 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf68593-e605-48df-bd45-505253c0d115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:06.499061 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-abf68593-e605-48df-bd45-505253c0d115 None None] MFA Rules not processed for user `eace14435db74925858cdce3fd70ca56`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:05:06.500303 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-abf68593-e605-48df-bd45-505253c0d115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:05:06.532403 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1099/2174] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:05:06 2022] POST /identity/v3/projects => generated 385 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:05:06.536919 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a59210-aadb-4059-9abb-de8c6c857c28 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:05:06.578346 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:05:06.602370 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1076/2175] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:05:06 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:05:06.608613 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf68593-e605-48df-bd45-505253c0d115 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:05:06.610624 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-abf68593-e605-48df-bd45-505253c0d115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:05:06.634540 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a59210-aadb-4059-9abb-de8c6c857c28 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:05:06.641933 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d2e109-19f3-4429-ac1d-18a2d9d0881a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:05:06.642629 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d2e109-19f3-4429-ac1d-18a2d9d0881a admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:06.643091 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d2e109-19f3-4429-ac1d-18a2d9d0881a admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:06.643376 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d2e109-19f3-4429-ac1d-18a2d9d0881a admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:06.644098 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d2e109-19f3-4429-ac1d-18a2d9d0881a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:05:06.646018 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d2e109-19f3-4429-ac1d-18a2d9d0881a admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:05:06.719226 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf68593-e605-48df-bd45-505253c0d115 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:05:06.778292 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1100/2176] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:05:06 2022] POST /identity/v3/users => generated 293 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:05:06.782295 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d2e109-19f3-4429-ac1d-18a2d9d0881a admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:05:06.789159 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774122a5-23a4-4c98-a974-686c24ea9066 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6cc95a563b941ac92ab283aa0d5a5d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:05:06.789159 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774122a5-23a4-4c98-a974-686c24ea9066 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:06.789159 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774122a5-23a4-4c98-a974-686c24ea9066 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:06.789159 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774122a5-23a4-4c98-a974-686c24ea9066 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:06.790763 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-774122a5-23a4-4c98-a974-686c24ea9066 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:05:06.796002 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d2e109-19f3-4429-ac1d-18a2d9d0881a admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:05:06.801276 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d52d927-5a43-4c18-b56b-b762cbb79654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:05:06.802242 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d52d927-5a43-4c18-b56b-b762cbb79654 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:06.803268 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d52d927-5a43-4c18-b56b-b762cbb79654 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:06.803665 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d52d927-5a43-4c18-b56b-b762cbb79654 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:06.807973 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d52d927-5a43-4c18-b56b-b762cbb79654 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:05:06.809642 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d52d927-5a43-4c18-b56b-b762cbb79654 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:05:06.843922 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1101/2177] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:05:06 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:05:06.860846 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d52d927-5a43-4c18-b56b-b762cbb79654 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:05:06.886177 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d52d927-5a43-4c18-b56b-b762cbb79654 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:05:06.892605 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04be5cbb-b022-4ad9-b12b-4f473caac8aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:05:06.894074 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04be5cbb-b022-4ad9-b12b-4f473caac8aa admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:06.894440 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04be5cbb-b022-4ad9-b12b-4f473caac8aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:06.894796 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04be5cbb-b022-4ad9-b12b-4f473caac8aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:06.896073 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04be5cbb-b022-4ad9-b12b-4f473caac8aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:05:06.898212 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04be5cbb-b022-4ad9-b12b-4f473caac8aa admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:05:06.917513 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1102/2178] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:05:06 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:05:06.919151 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04be5cbb-b022-4ad9-b12b-4f473caac8aa admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:05:06.934003 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04be5cbb-b022-4ad9-b12b-4f473caac8aa admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:05:06.937205 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774122a5-23a4-4c98-a974-686c24ea9066 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:05:06.938874 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774122a5-23a4-4c98-a974-686c24ea9066 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:05:06.939041 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabc7188-d71e-4c3c-b543-d3d4e896c799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:05:06.941642 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabc7188-d71e-4c3c-b543-d3d4e896c799 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:06.941948 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabc7188-d71e-4c3c-b543-d3d4e896c799 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:06.942233 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabc7188-d71e-4c3c-b543-d3d4e896c799 admin admin] PATH_INFO: `/v3/projects/ef745aade6c44699b924d7badc1500f7/users/35122a3ebfaa4915a6489d5bb8daaaed/roles/43106c664e1b40de984c0c62701149c8` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:06.953276 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1077/2179] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:05:06 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 10 13:05:06.967696 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabc7188-d71e-4c3c-b543-d3d4e896c799 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef745aade6c44699b924d7badc1500f7, user_id=35122a3ebfaa4915a6489d5bb8daaaed, role_id=43106c664e1b40de984c0c62701149c8)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:05:06.971220 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabc7188-d71e-4c3c-b543-d3d4e896c799 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:05:07.073994 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1103/2180] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:05:06 2022] PUT /identity/v3/projects/ef745aade6c44699b924d7badc1500f7/users/35122a3ebfaa4915a6489d5bb8daaaed/roles/43106c664e1b40de984c0c62701149c8 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 10 13:05:07.084206 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02e0136-7f7c-4c2c-ae7d-9977f3140097 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:07.084497 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02e0136-7f7c-4c2c-ae7d-9977f3140097 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:07.084778 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02e0136-7f7c-4c2c-ae7d-9977f3140097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:07.116818 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-c02e0136-7f7c-4c2c-ae7d-9977f3140097 None None] MFA Rules not processed for user `35122a3ebfaa4915a6489d5bb8daaaed`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:05:07.125623 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c02e0136-7f7c-4c2c-ae7d-9977f3140097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:05:07.247383 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1078/2181] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:05:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:05:07.251577 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabc7188-d71e-4c3c-b543-d3d4e896c799 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:05:07.265033 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:05:07.340035 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabc7188-d71e-4c3c-b543-d3d4e896c799 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:05:07.346192 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649e5eee-f425-4a9c-81ad-999b57607525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:05:07.350834 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649e5eee-f425-4a9c-81ad-999b57607525 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:07.350834 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649e5eee-f425-4a9c-81ad-999b57607525 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:07.350834 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649e5eee-f425-4a9c-81ad-999b57607525 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:07.350834 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649e5eee-f425-4a9c-81ad-999b57607525 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:05:07.353552 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649e5eee-f425-4a9c-81ad-999b57607525 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:05:07.424543 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:05:07.451246 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1104/2182] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:05:07 2022] POST /identity/v3/projects => generated 383 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:05:07.452600 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02e0136-7f7c-4c2c-ae7d-9977f3140097 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:05:07.469933 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02e0136-7f7c-4c2c-ae7d-9977f3140097 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:05:07.571351 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc026da-0a8e-4a00-ae20-8ffde58c742b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:05:07.572319 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc026da-0a8e-4a00-ae20-8ffde58c742b admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:07.572869 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc026da-0a8e-4a00-ae20-8ffde58c742b admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:07.573232 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc026da-0a8e-4a00-ae20-8ffde58c742b admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:07.578687 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc026da-0a8e-4a00-ae20-8ffde58c742b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:05:07.578687 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc026da-0a8e-4a00-ae20-8ffde58c742b admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:05:07.640882 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1079/2183] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:05:07 2022] POST /identity/v3/users => generated 298 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:05:07.642212 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649e5eee-f425-4a9c-81ad-999b57607525 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:05:07.682070 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649e5eee-f425-4a9c-81ad-999b57607525 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:05:07.697831 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63bfbb4-82e3-4db1-bc64-041c9892f1d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:05:07.699276 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63bfbb4-82e3-4db1-bc64-041c9892f1d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:07.711011 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63bfbb4-82e3-4db1-bc64-041c9892f1d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:07.711011 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63bfbb4-82e3-4db1-bc64-041c9892f1d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:07.711011 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63bfbb4-82e3-4db1-bc64-041c9892f1d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:05:07.711011 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63bfbb4-82e3-4db1-bc64-041c9892f1d8 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:05:07.718355 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1105/2184] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:05:07 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:05:07.722519 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc026da-0a8e-4a00-ae20-8ffde58c742b admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:05:07.739801 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc026da-0a8e-4a00-ae20-8ffde58c742b admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:05:07.751356 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cbc0cb-d178-48d7-b564-06d89d988af2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:05:07.752855 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cbc0cb-d178-48d7-b564-06d89d988af2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:07.753561 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cbc0cb-d178-48d7-b564-06d89d988af2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:07.753937 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cbc0cb-d178-48d7-b564-06d89d988af2 admin admin] PATH_INFO: `/v3/projects/fd3f00b0e53f4216bef4673be036c44d/users/74ac349d35d54fe3a278048c5cc8d37e/roles/b43d5a8247e244869c2c70f986f42e8a` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:07.776167 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cbc0cb-d178-48d7-b564-06d89d988af2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd3f00b0e53f4216bef4673be036c44d, user_id=74ac349d35d54fe3a278048c5cc8d37e, role_id=b43d5a8247e244869c2c70f986f42e8a)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:05:07.779440 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cbc0cb-d178-48d7-b564-06d89d988af2 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:05:07.905077 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1080/2185] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:05:07 2022] PUT /identity/v3/projects/fd3f00b0e53f4216bef4673be036c44d/users/74ac349d35d54fe3a278048c5cc8d37e/roles/b43d5a8247e244869c2c70f986f42e8a => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:05:07.909197 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63bfbb4-82e3-4db1-bc64-041c9892f1d8 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:05:07.920366 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63bfbb4-82e3-4db1-bc64-041c9892f1d8 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:05:07.990090 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ec0bf0-2266-4078-9d2d-0134cf41ffd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:05:07.990773 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ec0bf0-2266-4078-9d2d-0134cf41ffd6 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:07.991061 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ec0bf0-2266-4078-9d2d-0134cf41ffd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:07.991341 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ec0bf0-2266-4078-9d2d-0134cf41ffd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:07.992451 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ec0bf0-2266-4078-9d2d-0134cf41ffd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:05:07.994310 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ec0bf0-2266-4078-9d2d-0134cf41ffd6 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:05:08.005666 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:05:08.013335 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1106/2186] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:05:07 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 10 13:05:08.015415 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cbc0cb-d178-48d7-b564-06d89d988af2 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:05:08.032359 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cbc0cb-d178-48d7-b564-06d89d988af2 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:05:08.038975 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8165e6ea-8235-4013-a5c1-63a44420ac32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:05:08.041061 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8165e6ea-8235-4013-a5c1-63a44420ac32 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:08.041499 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8165e6ea-8235-4013-a5c1-63a44420ac32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:08.041942 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8165e6ea-8235-4013-a5c1-63a44420ac32 admin admin] PATH_INFO: `/v3/projects/fd3f00b0e53f4216bef4673be036c44d/users/74ac349d35d54fe3a278048c5cc8d37e/roles/b43d5a8247e244869c2c70f986f42e8a` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:08.044709 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8165e6ea-8235-4013-a5c1-63a44420ac32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd3f00b0e53f4216bef4673be036c44d, user_id=74ac349d35d54fe3a278048c5cc8d37e, role_id=b43d5a8247e244869c2c70f986f42e8a)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:05:08.049311 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8165e6ea-8235-4013-a5c1-63a44420ac32 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:05:08.077802 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1081/2187] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:05:08 2022] PUT /identity/v3/projects/fd3f00b0e53f4216bef4673be036c44d/users/74ac349d35d54fe3a278048c5cc8d37e/roles/b43d5a8247e244869c2c70f986f42e8a => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:05:08.089541 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb48276-d15e-4b29-92ef-e581f63e0b05 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:08.090730 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb48276-d15e-4b29-92ef-e581f63e0b05 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:08.091116 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb48276-d15e-4b29-92ef-e581f63e0b05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:08.141314 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-dbb48276-d15e-4b29-92ef-e581f63e0b05 None None] MFA Rules not processed for user `74ac349d35d54fe3a278048c5cc8d37e`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:05:08.147597 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-dbb48276-d15e-4b29-92ef-e581f63e0b05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:05:08.315169 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1107/2188] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:05:08 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:05:08.335545 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8165e6ea-8235-4013-a5c1-63a44420ac32 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:05:08.398541 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:05:08.438581 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8165e6ea-8235-4013-a5c1-63a44420ac32 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:05:08.444562 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef54581-0d9e-451d-8311-0a7f18f0d1fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6cc95a563b941ac92ab283aa0d5a5d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:05:08.446058 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef54581-0d9e-451d-8311-0a7f18f0d1fe service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:08.448425 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef54581-0d9e-451d-8311-0a7f18f0d1fe service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:08.448425 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef54581-0d9e-451d-8311-0a7f18f0d1fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:08.575298 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef54581-0d9e-451d-8311-0a7f18f0d1fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:05:08.576835 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef54581-0d9e-451d-8311-0a7f18f0d1fe service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:05:08.586005 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1082/2189] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:05:08 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:05:09.843758 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb48276-d15e-4b29-92ef-e581f63e0b05 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:05:09.845252 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-dbb48276-d15e-4b29-92ef-e581f63e0b05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:05:09.963594 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb48276-d15e-4b29-92ef-e581f63e0b05 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:05:09.970088 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e21661-ea84-477d-a0ac-7c4a9f5a98bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:05:09.973762 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e21661-ea84-477d-a0ac-7c4a9f5a98bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:09.973762 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e21661-ea84-477d-a0ac-7c4a9f5a98bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:09.973762 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e21661-ea84-477d-a0ac-7c4a9f5a98bf admin admin] PATH_INFO: `/v3/users/3dcd9c7df0974b068d6017e85dc85a90` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:09.974326 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e21661-ea84-477d-a0ac-7c4a9f5a98bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=3dcd9c7df0974b068d6017e85dc85a90)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:05:09.979274 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e21661-ea84-477d-a0ac-7c4a9f5a98bf admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:05:10.252651 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef54581-0d9e-451d-8311-0a7f18f0d1fe service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:05:10.264654 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-84e21661-ea84-477d-a0ac-7c4a9f5a98bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3dcd9c7df0974b068d6017e85dc85a90'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:10.268976 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef54581-0d9e-451d-8311-0a7f18f0d1fe service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:05:10.274250 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5b1511-f6d7-4ea1-aefa-ec5d63b54c84 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6cc95a563b941ac92ab283aa0d5a5d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:05:10.275613 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5b1511-f6d7-4ea1-aefa-ec5d63b54c84 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:10.284061 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5b1511-f6d7-4ea1-aefa-ec5d63b54c84 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:10.284061 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5b1511-f6d7-4ea1-aefa-ec5d63b54c84 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:10.284350 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-84e21661-ea84-477d-a0ac-7c4a9f5a98bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3dcd9c7df0974b068d6017e85dc85a90'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:10.347369 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5b1511-f6d7-4ea1-aefa-ec5d63b54c84 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:05:10.348624 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5b1511-f6d7-4ea1-aefa-ec5d63b54c84 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:05:10.358928 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1083/2190] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:05:10 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 10 13:05:10.373862 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-84e21661-ea84-477d-a0ac-7c4a9f5a98bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3dcd9c7df0974b068d6017e85dc85a90'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:10.375055 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-84e21661-ea84-477d-a0ac-7c4a9f5a98bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3dcd9c7df0974b068d6017e85dc85a90'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:10.391344 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:05:10.393360 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-84e21661-ea84-477d-a0ac-7c4a9f5a98bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3dcd9c7df0974b068d6017e85dc85a90'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:10.406830 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1108/2191] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:05:09 2022] DELETE /identity/v3/users/3dcd9c7df0974b068d6017e85dc85a90 => generated 0 bytes in 560 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:05:10.408639 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3190d9-82fe-48c5-87ef-21e9350935ef None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:10.408639 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3190d9-82fe-48c5-87ef-21e9350935ef None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:10.408849 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3190d9-82fe-48c5-87ef-21e9350935ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:10.469233 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-bb3190d9-82fe-48c5-87ef-21e9350935ef None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:05:10.470812 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-bb3190d9-82fe-48c5-87ef-21e9350935ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:05:10.579792 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1084/2192] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:05:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:05:10.590331 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e21661-ea84-477d-a0ac-7c4a9f5a98bf admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:05:10.591763 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-84e21661-ea84-477d-a0ac-7c4a9f5a98bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:05:10.672320 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e21661-ea84-477d-a0ac-7c4a9f5a98bf admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:05:10.676083 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7b4608-5297-437d-99cf-b59f60cf4097 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:05:10.676732 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7b4608-5297-437d-99cf-b59f60cf4097 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:10.677016 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7b4608-5297-437d-99cf-b59f60cf4097 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:10.677455 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7b4608-5297-437d-99cf-b59f60cf4097 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:10.687310 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7b4608-5297-437d-99cf-b59f60cf4097 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:05:10.688953 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7b4608-5297-437d-99cf-b59f60cf4097 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:05:10.703219 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1109/2193] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:05:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:05:11.261813 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3190d9-82fe-48c5-87ef-21e9350935ef None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:05:11.263806 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-bb3190d9-82fe-48c5-87ef-21e9350935ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:05:11.278314 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3190d9-82fe-48c5-87ef-21e9350935ef None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:05:11.287556 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9668531c-55ad-4081-b1dc-49297c2e07f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:05:11.287556 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9668531c-55ad-4081-b1dc-49297c2e07f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:11.287556 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9668531c-55ad-4081-b1dc-49297c2e07f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:11.287556 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9668531c-55ad-4081-b1dc-49297c2e07f2 admin admin] PATH_INFO: `/v3/projects/16dc27d7792c4f92894ed9f677880bc3` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:11.287887 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9668531c-55ad-4081-b1dc-49297c2e07f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=16dc27d7792c4f92894ed9f677880bc3)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:05:11.289800 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9668531c-55ad-4081-b1dc-49297c2e07f2 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:05:11.307289 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:05:11.430900 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-9668531c-55ad-4081-b1dc-49297c2e07f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16dc27d7792c4f92894ed9f677880bc3', 'request_id': 'req-9668531c-55ad-4081-b1dc-49297c2e07f2'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:11.437438 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-9668531c-55ad-4081-b1dc-49297c2e07f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16dc27d7792c4f92894ed9f677880bc3', 'request_id': 'req-9668531c-55ad-4081-b1dc-49297c2e07f2'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:11.445626 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-9668531c-55ad-4081-b1dc-49297c2e07f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16dc27d7792c4f92894ed9f677880bc3', 'request_id': 'req-9668531c-55ad-4081-b1dc-49297c2e07f2'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:11.557643 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-9668531c-55ad-4081-b1dc-49297c2e07f2 admin admin] The token cache is being invalidate because project 16dc27d7792c4f92894ed9f677880bc3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:05:11.557643 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-9668531c-55ad-4081-b1dc-49297c2e07f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:11.560689 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-9668531c-55ad-4081-b1dc-49297c2e07f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:11.565289 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1085/2194] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:05:11 2022] DELETE /identity/v3/projects/16dc27d7792c4f92894ed9f677880bc3 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:05:11.973898 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7b4608-5297-437d-99cf-b59f60cf4097 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:05:11.975810 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ef7b4608-5297-437d-99cf-b59f60cf4097 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:05:12.002372 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414755ce-868e-48b9-8f64-4c0b05d1ffda None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:12.002970 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414755ce-868e-48b9-8f64-4c0b05d1ffda None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:12.003461 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414755ce-868e-48b9-8f64-4c0b05d1ffda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:12.074158 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-414755ce-868e-48b9-8f64-4c0b05d1ffda None None] MFA Rules not processed for user `869c7d82fde44c4083eeb51dc91b2b61`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:05:12.077019 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-414755ce-868e-48b9-8f64-4c0b05d1ffda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:05:12.080256 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7b4608-5297-437d-99cf-b59f60cf4097 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:05:12.173927 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e39fca-8d4e-47d8-8dd4-ab4d6f5e224f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6cc95a563b941ac92ab283aa0d5a5d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:05:12.174713 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e39fca-8d4e-47d8-8dd4-ab4d6f5e224f service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:12.175003 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e39fca-8d4e-47d8-8dd4-ab4d6f5e224f service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:12.175341 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e39fca-8d4e-47d8-8dd4-ab4d6f5e224f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:12.185490 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-05e39fca-8d4e-47d8-8dd4-ab4d6f5e224f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:05:12.186854 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1086/2195] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:05:11 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 10 13:05:12.186854 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414755ce-868e-48b9-8f64-4c0b05d1ffda None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:05:12.186854 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-414755ce-868e-48b9-8f64-4c0b05d1ffda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:05:12.213301 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414755ce-868e-48b9-8f64-4c0b05d1ffda None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:05:12.219797 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37486d7e-741d-4553-bf73-d59766d6290b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6cc95a563b941ac92ab283aa0d5a5d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:05:12.226283 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37486d7e-741d-4553-bf73-d59766d6290b service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:12.226713 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37486d7e-741d-4553-bf73-d59766d6290b service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:12.227002 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37486d7e-741d-4553-bf73-d59766d6290b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:12.229205 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-37486d7e-741d-4553-bf73-d59766d6290b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:05:12.272813 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:05:12.289250 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e39fca-8d4e-47d8-8dd4-ab4d6f5e224f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:05:12.289250 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e39fca-8d4e-47d8-8dd4-ab4d6f5e224f service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:05:12.309251 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1110/2196] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:05:11 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:05:12.320830 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e39fca-8d4e-47d8-8dd4-ab4d6f5e224f service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:05:12.325479 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-05e39fca-8d4e-47d8-8dd4-ab4d6f5e224f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:05:12.328790 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37486d7e-741d-4553-bf73-d59766d6290b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:05:12.330309 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37486d7e-741d-4553-bf73-d59766d6290b service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:05:12.344646 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1087/2197] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:05:12 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:05:12.480628 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e39fca-8d4e-47d8-8dd4-ab4d6f5e224f service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:05:12.486324 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185f7711-6f11-42ef-81dd-b62ecd395218 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bd6caa461e84100a3d5fe5af5d80f3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:05:12.486974 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185f7711-6f11-42ef-81dd-b62ecd395218 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:12.487258 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185f7711-6f11-42ef-81dd-b62ecd395218 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:12.487533 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185f7711-6f11-42ef-81dd-b62ecd395218 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:12.489485 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-185f7711-6f11-42ef-81dd-b62ecd395218 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:05:12.566974 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185f7711-6f11-42ef-81dd-b62ecd395218 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:05:12.568677 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185f7711-6f11-42ef-81dd-b62ecd395218 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:05:12.581870 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1111/2198] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:05:12 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:05:14.204237 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37486d7e-741d-4553-bf73-d59766d6290b service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:05:14.206364 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-37486d7e-741d-4553-bf73-d59766d6290b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:05:14.224641 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37486d7e-741d-4553-bf73-d59766d6290b service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:05:14.249522 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6c4df7-65c3-449f-bd40-79ede6dc2354 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6cc95a563b941ac92ab283aa0d5a5d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:05:14.250511 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6c4df7-65c3-449f-bd40-79ede6dc2354 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:14.250973 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6c4df7-65c3-449f-bd40-79ede6dc2354 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:14.251404 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6c4df7-65c3-449f-bd40-79ede6dc2354 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:14.253476 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-8d6c4df7-65c3-449f-bd40-79ede6dc2354 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:05:14.272719 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:05:14.343212 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6c4df7-65c3-449f-bd40-79ede6dc2354 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:05:14.344702 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6c4df7-65c3-449f-bd40-79ede6dc2354 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:05:14.360250 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1088/2199] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:05:14 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:05:15.666067 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ded74b-f5ee-4b1c-a42e-4ed2cdc044d6 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:15.666911 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ded74b-f5ee-4b1c-a42e-4ed2cdc044d6 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:15.667162 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ded74b-f5ee-4b1c-a42e-4ed2cdc044d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:15.703922 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-c6ded74b-f5ee-4b1c-a42e-4ed2cdc044d6 None None] MFA Rules not processed for user `a72b5e893615479ea3fe1838d07496ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:05:15.703922 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c6ded74b-f5ee-4b1c-a42e-4ed2cdc044d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:05:15.718532 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1112/2200] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:05:15 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:05:18.276307 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6c4df7-65c3-449f-bd40-79ede6dc2354 service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:05:18.277752 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-8d6c4df7-65c3-449f-bd40-79ede6dc2354 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:05:18.348488 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6c4df7-65c3-449f-bd40-79ede6dc2354 service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:05:18.356282 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306ca6b3-6174-4784-9b7b-fe2573404214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:05:18.357171 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306ca6b3-6174-4784-9b7b-fe2573404214 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:18.357442 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306ca6b3-6174-4784-9b7b-fe2573404214 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:18.357721 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306ca6b3-6174-4784-9b7b-fe2573404214 admin admin] PATH_INFO: `/v3/users/ea3ad28e5b9d48c9acbf4ea3d554fb1f` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:18.358916 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306ca6b3-6174-4784-9b7b-fe2573404214 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea3ad28e5b9d48c9acbf4ea3d554fb1f)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:05:18.361091 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306ca6b3-6174-4784-9b7b-fe2573404214 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:05:18.501780 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-306ca6b3-6174-4784-9b7b-fe2573404214 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea3ad28e5b9d48c9acbf4ea3d554fb1f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:18.509853 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-306ca6b3-6174-4784-9b7b-fe2573404214 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea3ad28e5b9d48c9acbf4ea3d554fb1f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:18.604671 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-306ca6b3-6174-4784-9b7b-fe2573404214 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea3ad28e5b9d48c9acbf4ea3d554fb1f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:18.605361 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-306ca6b3-6174-4784-9b7b-fe2573404214 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea3ad28e5b9d48c9acbf4ea3d554fb1f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:18.621125 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-306ca6b3-6174-4784-9b7b-fe2573404214 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea3ad28e5b9d48c9acbf4ea3d554fb1f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:18.627327 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1089/2201] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:05:18 2022] DELETE /identity/v3/users/ea3ad28e5b9d48c9acbf4ea3d554fb1f => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:05:18.629114 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e3de82-ac8c-47c6-9f80-d7f0a33d8c3a None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:18.629565 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e3de82-ac8c-47c6-9f80-d7f0a33d8c3a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:18.629666 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e3de82-ac8c-47c6-9f80-d7f0a33d8c3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:18.647978 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-e2e3de82-ac8c-47c6-9f80-d7f0a33d8c3a None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:05:18.649811 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e2e3de82-ac8c-47c6-9f80-d7f0a33d8c3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:05:18.709804 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1113/2202] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:05:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:05:18.728742 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306ca6b3-6174-4784-9b7b-fe2573404214 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:05:18.730224 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-306ca6b3-6174-4784-9b7b-fe2573404214 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:05:18.769016 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:05:18.794820 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306ca6b3-6174-4784-9b7b-fe2573404214 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:05:18.798506 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4210ffa9-a0a5-48bb-b043-54e379e413df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:05:18.799193 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4210ffa9-a0a5-48bb-b043-54e379e413df service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:18.799596 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4210ffa9-a0a5-48bb-b043-54e379e413df service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:18.799896 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4210ffa9-a0a5-48bb-b043-54e379e413df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:18.809341 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4210ffa9-a0a5-48bb-b043-54e379e413df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:05:18.811254 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4210ffa9-a0a5-48bb-b043-54e379e413df service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:05:18.823084 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1090/2203] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:05:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:05:19.300436 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e3de82-ac8c-47c6-9f80-d7f0a33d8c3a None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:05:19.301926 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e2e3de82-ac8c-47c6-9f80-d7f0a33d8c3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:05:19.315810 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e3de82-ac8c-47c6-9f80-d7f0a33d8c3a None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:05:19.323175 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0233678f-9b65-41ad-a1be-abaafc371f34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:05:19.324077 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0233678f-9b65-41ad-a1be-abaafc371f34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:19.324362 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0233678f-9b65-41ad-a1be-abaafc371f34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:19.324780 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0233678f-9b65-41ad-a1be-abaafc371f34 admin admin] PATH_INFO: `/v3/projects/ea02b87433cf43f6824bfcbd8245142a` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:19.326080 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0233678f-9b65-41ad-a1be-abaafc371f34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea02b87433cf43f6824bfcbd8245142a)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:05:19.328151 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0233678f-9b65-41ad-a1be-abaafc371f34 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:05:19.344419 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:05:19.487435 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0233678f-9b65-41ad-a1be-abaafc371f34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea02b87433cf43f6824bfcbd8245142a', 'request_id': 'req-0233678f-9b65-41ad-a1be-abaafc371f34'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:19.493423 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0233678f-9b65-41ad-a1be-abaafc371f34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea02b87433cf43f6824bfcbd8245142a', 'request_id': 'req-0233678f-9b65-41ad-a1be-abaafc371f34'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:19.499638 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0233678f-9b65-41ad-a1be-abaafc371f34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea02b87433cf43f6824bfcbd8245142a', 'request_id': 'req-0233678f-9b65-41ad-a1be-abaafc371f34'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:19.583526 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0233678f-9b65-41ad-a1be-abaafc371f34 admin admin] The token cache is being invalidate because project ea02b87433cf43f6824bfcbd8245142a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:05:19.583820 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0233678f-9b65-41ad-a1be-abaafc371f34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:19.584543 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0233678f-9b65-41ad-a1be-abaafc371f34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:19.586284 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1114/2204] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:05:19 2022] DELETE /identity/v3/projects/ea02b87433cf43f6824bfcbd8245142a => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:05:19.590563 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4210ffa9-a0a5-48bb-b043-54e379e413df service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:05:19.600368 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4210ffa9-a0a5-48bb-b043-54e379e413df service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:05:19.661109 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2677b62-67a2-4880-b1a6-1df0edcc818c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:05:19.662024 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2677b62-67a2-4880-b1a6-1df0edcc818c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:19.662319 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2677b62-67a2-4880-b1a6-1df0edcc818c admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:19.662771 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2677b62-67a2-4880-b1a6-1df0edcc818c admin admin] PATH_INFO: `/v3/users/9acccd4e69014cbeb566f94a8a579fec` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:19.664067 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2677b62-67a2-4880-b1a6-1df0edcc818c admin admin] RBAC: Authorizing `identity:delete_user(user_id=9acccd4e69014cbeb566f94a8a579fec)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:05:19.666623 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2677b62-67a2-4880-b1a6-1df0edcc818c admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:05:19.877573 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-e2677b62-67a2-4880-b1a6-1df0edcc818c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9acccd4e69014cbeb566f94a8a579fec'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:19.890182 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-e2677b62-67a2-4880-b1a6-1df0edcc818c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9acccd4e69014cbeb566f94a8a579fec'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:19.966680 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-e2677b62-67a2-4880-b1a6-1df0edcc818c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9acccd4e69014cbeb566f94a8a579fec'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:19.967303 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-e2677b62-67a2-4880-b1a6-1df0edcc818c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9acccd4e69014cbeb566f94a8a579fec'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:19.989077 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-e2677b62-67a2-4880-b1a6-1df0edcc818c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9acccd4e69014cbeb566f94a8a579fec'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:19.995453 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1091/2205] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:05:19 2022] DELETE /identity/v3/users/9acccd4e69014cbeb566f94a8a579fec => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:05:20.252045 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0233678f-9b65-41ad-a1be-abaafc371f34 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:05:20.253570 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0233678f-9b65-41ad-a1be-abaafc371f34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:05:20.267353 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0233678f-9b65-41ad-a1be-abaafc371f34 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:05:20.327809 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5796cf-dc81-4f5f-9983-940acf9ce88e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:05:20.328777 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5796cf-dc81-4f5f-9983-940acf9ce88e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:20.329038 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5796cf-dc81-4f5f-9983-940acf9ce88e admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:20.329320 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5796cf-dc81-4f5f-9983-940acf9ce88e admin admin] PATH_INFO: `/v3/users/5da72979b3774c62a6963e0b641923bf` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:20.330671 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5796cf-dc81-4f5f-9983-940acf9ce88e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5da72979b3774c62a6963e0b641923bf)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:05:20.332756 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5796cf-dc81-4f5f-9983-940acf9ce88e admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:05:20.410131 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2677b62-67a2-4880-b1a6-1df0edcc818c admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:05:20.411629 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e2677b62-67a2-4880-b1a6-1df0edcc818c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:05:20.424606 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2677b62-67a2-4880-b1a6-1df0edcc818c admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:05:20.433104 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6174499d-fe5c-4698-aa51-3c9a631c4a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:05:20.433966 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6174499d-fe5c-4698-aa51-3c9a631c4a55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:20.434246 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6174499d-fe5c-4698-aa51-3c9a631c4a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:20.434580 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6174499d-fe5c-4698-aa51-3c9a631c4a55 admin admin] PATH_INFO: `/v3/projects/d17d6695ebdd4302be23304b37a5646a` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:20.435712 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6174499d-fe5c-4698-aa51-3c9a631c4a55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d17d6695ebdd4302be23304b37a5646a)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:05:20.441011 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6174499d-fe5c-4698-aa51-3c9a631c4a55 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:05:20.524429 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-9e5796cf-dc81-4f5f-9983-940acf9ce88e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5da72979b3774c62a6963e0b641923bf'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:20.531595 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-9e5796cf-dc81-4f5f-9983-940acf9ce88e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5da72979b3774c62a6963e0b641923bf'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:20.554588 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:05:20.556543 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6174499d-fe5c-4698-aa51-3c9a631c4a55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd17d6695ebdd4302be23304b37a5646a', 'request_id': 'req-6174499d-fe5c-4698-aa51-3c9a631c4a55'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:20.562683 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6174499d-fe5c-4698-aa51-3c9a631c4a55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd17d6695ebdd4302be23304b37a5646a', 'request_id': 'req-6174499d-fe5c-4698-aa51-3c9a631c4a55'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:20.569969 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6174499d-fe5c-4698-aa51-3c9a631c4a55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd17d6695ebdd4302be23304b37a5646a', 'request_id': 'req-6174499d-fe5c-4698-aa51-3c9a631c4a55'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:20.607845 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-9e5796cf-dc81-4f5f-9983-940acf9ce88e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5da72979b3774c62a6963e0b641923bf'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:20.608573 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-9e5796cf-dc81-4f5f-9983-940acf9ce88e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5da72979b3774c62a6963e0b641923bf'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:20.621407 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-9e5796cf-dc81-4f5f-9983-940acf9ce88e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5da72979b3774c62a6963e0b641923bf'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:20.627354 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1115/2206] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:05:20 2022] DELETE /identity/v3/users/5da72979b3774c62a6963e0b641923bf => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:05:20.631542 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea96d5a6-2f99-41d5-8ff2-63f2038f7c41 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:20.631854 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea96d5a6-2f99-41d5-8ff2-63f2038f7c41 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:20.632168 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea96d5a6-2f99-41d5-8ff2-63f2038f7c41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:20.651167 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-ea96d5a6-2f99-41d5-8ff2-63f2038f7c41 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:05:20.652345 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ea96d5a6-2f99-41d5-8ff2-63f2038f7c41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:05:20.669855 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:05:20.711079 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1116/2207] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:05:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 10 13:05:20.714873 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6174499d-fe5c-4698-aa51-3c9a631c4a55 admin admin] The token cache is being invalidate because project d17d6695ebdd4302be23304b37a5646a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:05:20.715330 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6174499d-fe5c-4698-aa51-3c9a631c4a55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:20.715777 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6174499d-fe5c-4698-aa51-3c9a631c4a55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:20.717335 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1092/2208] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:05:20 2022] DELETE /identity/v3/projects/d17d6695ebdd4302be23304b37a5646a => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:05:20.720921 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6174499d-fe5c-4698-aa51-3c9a631c4a55 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:05:20.721082 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea96d5a6-2f99-41d5-8ff2-63f2038f7c41 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:05:20.722345 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ea96d5a6-2f99-41d5-8ff2-63f2038f7c41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:05:20.722479 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-6174499d-fe5c-4698-aa51-3c9a631c4a55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:05:20.791286 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea96d5a6-2f99-41d5-8ff2-63f2038f7c41 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:05:20.795476 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b97ed12-e2d1-47b9-bffe-76485e6770a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:05:20.796237 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b97ed12-e2d1-47b9-bffe-76485e6770a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:20.796594 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b97ed12-e2d1-47b9-bffe-76485e6770a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:20.796951 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b97ed12-e2d1-47b9-bffe-76485e6770a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:20.797110 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6174499d-fe5c-4698-aa51-3c9a631c4a55 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:05:20.800049 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-4b97ed12-e2d1-47b9-bffe-76485e6770a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:05:20.802419 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7fe91a-8063-4df7-ac4b-6b92552a1c42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:05:20.803304 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7fe91a-8063-4df7-ac4b-6b92552a1c42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:20.803607 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7fe91a-8063-4df7-ac4b-6b92552a1c42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:20.803893 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7fe91a-8063-4df7-ac4b-6b92552a1c42 admin admin] PATH_INFO: `/v3/users/eace14435db74925858cdce3fd70ca56` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:20.805752 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7fe91a-8063-4df7-ac4b-6b92552a1c42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eace14435db74925858cdce3fd70ca56)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:05:20.813212 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7fe91a-8063-4df7-ac4b-6b92552a1c42 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:05:20.815737 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b97ed12-e2d1-47b9-bffe-76485e6770a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:05:20.817385 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b97ed12-e2d1-47b9-bffe-76485e6770a2 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:05:20.829353 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1117/2209] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:05:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:05:21.014794 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-fc7fe91a-8063-4df7-ac4b-6b92552a1c42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eace14435db74925858cdce3fd70ca56'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:21.022202 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-fc7fe91a-8063-4df7-ac4b-6b92552a1c42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eace14435db74925858cdce3fd70ca56'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:21.072364 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-fc7fe91a-8063-4df7-ac4b-6b92552a1c42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eace14435db74925858cdce3fd70ca56'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:21.073025 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-fc7fe91a-8063-4df7-ac4b-6b92552a1c42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eace14435db74925858cdce3fd70ca56'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:21.086165 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-fc7fe91a-8063-4df7-ac4b-6b92552a1c42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eace14435db74925858cdce3fd70ca56'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:21.094657 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1093/2210] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:05:20 2022] DELETE /identity/v3/users/eace14435db74925858cdce3fd70ca56 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:05:21.543818 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b97ed12-e2d1-47b9-bffe-76485e6770a2 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:05:21.545486 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-4b97ed12-e2d1-47b9-bffe-76485e6770a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:05:21.566109 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b97ed12-e2d1-47b9-bffe-76485e6770a2 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:05:21.627168 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc1dfbd-c73c-46bd-884e-777e47d37427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:05:21.628091 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc1dfbd-c73c-46bd-884e-777e47d37427 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:21.628525 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc1dfbd-c73c-46bd-884e-777e47d37427 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:21.628852 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc1dfbd-c73c-46bd-884e-777e47d37427 admin admin] PATH_INFO: `/v3/projects/3e56e08954de404a8ac6db8fdfcafe76` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:21.630160 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc1dfbd-c73c-46bd-884e-777e47d37427 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e56e08954de404a8ac6db8fdfcafe76)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:05:21.636046 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc1dfbd-c73c-46bd-884e-777e47d37427 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:05:21.763169 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3bc1dfbd-c73c-46bd-884e-777e47d37427 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e56e08954de404a8ac6db8fdfcafe76', 'request_id': 'req-3bc1dfbd-c73c-46bd-884e-777e47d37427'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:21.768983 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3bc1dfbd-c73c-46bd-884e-777e47d37427 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e56e08954de404a8ac6db8fdfcafe76', 'request_id': 'req-3bc1dfbd-c73c-46bd-884e-777e47d37427'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:21.776505 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3bc1dfbd-c73c-46bd-884e-777e47d37427 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e56e08954de404a8ac6db8fdfcafe76', 'request_id': 'req-3bc1dfbd-c73c-46bd-884e-777e47d37427'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:21.780149 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:05:21.859071 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3bc1dfbd-c73c-46bd-884e-777e47d37427 admin admin] The token cache is being invalidate because project 3e56e08954de404a8ac6db8fdfcafe76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:05:21.859071 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3bc1dfbd-c73c-46bd-884e-777e47d37427 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:21.864503 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-3bc1dfbd-c73c-46bd-884e-777e47d37427 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:21.864503 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1118/2211] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:05:21 2022] DELETE /identity/v3/projects/3e56e08954de404a8ac6db8fdfcafe76 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:05:21.865736 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7fe91a-8063-4df7-ac4b-6b92552a1c42 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:05:21.868328 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-fc7fe91a-8063-4df7-ac4b-6b92552a1c42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:05:21.890825 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc1dfbd-c73c-46bd-884e-777e47d37427 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:05:21.892465 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7fe91a-8063-4df7-ac4b-6b92552a1c42 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:05:21.894586 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3bc1dfbd-c73c-46bd-884e-777e47d37427 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:05:21.967259 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:05:21.976004 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d88e8d0-3ca4-4c55-b23d-ee3a060ade30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:05:21.977453 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d88e8d0-3ca4-4c55-b23d-ee3a060ade30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:21.977829 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d88e8d0-3ca4-4c55-b23d-ee3a060ade30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:21.979326 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d88e8d0-3ca4-4c55-b23d-ee3a060ade30 admin admin] PATH_INFO: `/v3/users/35122a3ebfaa4915a6489d5bb8daaaed` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:21.979549 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d88e8d0-3ca4-4c55-b23d-ee3a060ade30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35122a3ebfaa4915a6489d5bb8daaaed)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:05:21.981765 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d88e8d0-3ca4-4c55-b23d-ee3a060ade30 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:05:21.984789 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc1dfbd-c73c-46bd-884e-777e47d37427 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:05:21.992696 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14b2069-1b88-44b1-8fa0-aa74a3f04cf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:05:21.995701 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14b2069-1b88-44b1-8fa0-aa74a3f04cf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:21.995701 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14b2069-1b88-44b1-8fa0-aa74a3f04cf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:21.995701 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14b2069-1b88-44b1-8fa0-aa74a3f04cf0 admin admin] PATH_INFO: `/v3/projects/26d4bc05d9db455e8f72be806503d06c` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:21.996005 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14b2069-1b88-44b1-8fa0-aa74a3f04cf0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=26d4bc05d9db455e8f72be806503d06c)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:05:21.998556 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14b2069-1b88-44b1-8fa0-aa74a3f04cf0 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:05:22.206836 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e14b2069-1b88-44b1-8fa0-aa74a3f04cf0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26d4bc05d9db455e8f72be806503d06c', 'request_id': 'req-e14b2069-1b88-44b1-8fa0-aa74a3f04cf0'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:22.207100 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-8d88e8d0-3ca4-4c55-b23d-ee3a060ade30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35122a3ebfaa4915a6489d5bb8daaaed'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:22.214231 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e14b2069-1b88-44b1-8fa0-aa74a3f04cf0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26d4bc05d9db455e8f72be806503d06c', 'request_id': 'req-e14b2069-1b88-44b1-8fa0-aa74a3f04cf0'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:22.215720 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-8d88e8d0-3ca4-4c55-b23d-ee3a060ade30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35122a3ebfaa4915a6489d5bb8daaaed'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:22.221946 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e14b2069-1b88-44b1-8fa0-aa74a3f04cf0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26d4bc05d9db455e8f72be806503d06c', 'request_id': 'req-e14b2069-1b88-44b1-8fa0-aa74a3f04cf0'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:22.320450 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-8d88e8d0-3ca4-4c55-b23d-ee3a060ade30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35122a3ebfaa4915a6489d5bb8daaaed'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:22.321642 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-8d88e8d0-3ca4-4c55-b23d-ee3a060ade30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35122a3ebfaa4915a6489d5bb8daaaed'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:22.333907 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-8d88e8d0-3ca4-4c55-b23d-ee3a060ade30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35122a3ebfaa4915a6489d5bb8daaaed'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:22.340370 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1094/2212] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:05:21 2022] DELETE /identity/v3/users/35122a3ebfaa4915a6489d5bb8daaaed => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:05:22.341587 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273826ee-1b34-42a8-b06e-188f9e009ed6 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:22.341875 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273826ee-1b34-42a8-b06e-188f9e009ed6 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:22.342152 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273826ee-1b34-42a8-b06e-188f9e009ed6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:22.360323 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-273826ee-1b34-42a8-b06e-188f9e009ed6 None None] MFA Rules not processed for user `e0b63da5bf0b4410acc9ff573064f8ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:05:22.361296 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-273826ee-1b34-42a8-b06e-188f9e009ed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:05:22.385162 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e14b2069-1b88-44b1-8fa0-aa74a3f04cf0 admin admin] The token cache is being invalidate because project 26d4bc05d9db455e8f72be806503d06c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:05:22.385486 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e14b2069-1b88-44b1-8fa0-aa74a3f04cf0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:22.386179 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-e14b2069-1b88-44b1-8fa0-aa74a3f04cf0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:22.388167 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1119/2213] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:05:21 2022] DELETE /identity/v3/projects/26d4bc05d9db455e8f72be806503d06c => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:05:22.392698 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c75527f-fde5-4bce-8f2e-b10d9c540e90 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:22.392698 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c75527f-fde5-4bce-8f2e-b10d9c540e90 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:22.393041 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c75527f-fde5-4bce-8f2e-b10d9c540e90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:22.425449 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1095/2214] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:05:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:05:22.450224 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-6c75527f-fde5-4bce-8f2e-b10d9c540e90 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:05:22.450224 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6c75527f-fde5-4bce-8f2e-b10d9c540e90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:05:22.511392 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1120/2215] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:05:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:05:22.520250 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb6dffc-3cd6-4558-9c48-0dd97c2226df None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:22.520551 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb6dffc-3cd6-4558-9c48-0dd97c2226df None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:22.520835 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb6dffc-3cd6-4558-9c48-0dd97c2226df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:22.542908 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-bfb6dffc-3cd6-4558-9c48-0dd97c2226df None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:05:22.544570 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-bfb6dffc-3cd6-4558-9c48-0dd97c2226df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:05:22.554407 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1096/2216] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:05:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:05:22.558636 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c75527f-fde5-4bce-8f2e-b10d9c540e90 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:05:22.568797 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c75527f-fde5-4bce-8f2e-b10d9c540e90 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:05:22.580943 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d0db3f-4a04-4ad4-8c54-07bedf4f0143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:05:22.581797 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d0db3f-4a04-4ad4-8c54-07bedf4f0143 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:22.582133 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d0db3f-4a04-4ad4-8c54-07bedf4f0143 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:22.582616 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d0db3f-4a04-4ad4-8c54-07bedf4f0143 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:22.583633 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d0db3f-4a04-4ad4-8c54-07bedf4f0143 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 13:05:22.584050 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d0db3f-4a04-4ad4-8c54-07bedf4f0143 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:05:22.585958 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d0db3f-4a04-4ad4-8c54-07bedf4f0143 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:05:22.602776 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1121/2217] 10.222.0.27 () {58 vars in 1152 bytes} [Fri Jun 10 13:05:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 13:05:22.604003 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb6dffc-3cd6-4558-9c48-0dd97c2226df None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:05:22.615443 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb6dffc-3cd6-4558-9c48-0dd97c2226df None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:05:22.621064 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351872de-d1c7-4336-ac18-5dcf3cff8e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:05:22.621743 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351872de-d1c7-4336-ac18-5dcf3cff8e93 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:22.622025 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351872de-d1c7-4336-ac18-5dcf3cff8e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:22.622300 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351872de-d1c7-4336-ac18-5dcf3cff8e93 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:22.622989 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351872de-d1c7-4336-ac18-5dcf3cff8e93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:05:22.624912 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351872de-d1c7-4336-ac18-5dcf3cff8e93 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:05:22.710085 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1097/2218] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:05:22 2022] POST /identity/v3/projects => generated 365 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 10 13:05:22.714497 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d0db3f-4a04-4ad4-8c54-07bedf4f0143 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:05:22.725868 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d0db3f-4a04-4ad4-8c54-07bedf4f0143 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:05:22.790018 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98389311-2fbe-4812-a7bc-02a7f68e778e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:05:22.792170 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98389311-2fbe-4812-a7bc-02a7f68e778e admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:22.792170 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98389311-2fbe-4812-a7bc-02a7f68e778e admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:22.792170 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98389311-2fbe-4812-a7bc-02a7f68e778e admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:22.792170 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98389311-2fbe-4812-a7bc-02a7f68e778e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:05:22.794256 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98389311-2fbe-4812-a7bc-02a7f68e778e admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:05:22.812205 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:05:22.897546 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1122/2219] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:05:22 2022] POST /identity/v3/users => generated 283 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:05:22.901636 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351872de-d1c7-4336-ac18-5dcf3cff8e93 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:05:22.912071 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351872de-d1c7-4336-ac18-5dcf3cff8e93 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:05:22.917393 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22661c5f-6139-4813-81ec-a43c7a704693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:05:22.918109 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22661c5f-6139-4813-81ec-a43c7a704693 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:22.918387 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22661c5f-6139-4813-81ec-a43c7a704693 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:22.918721 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22661c5f-6139-4813-81ec-a43c7a704693 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:22.919384 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22661c5f-6139-4813-81ec-a43c7a704693 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:05:22.921050 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22661c5f-6139-4813-81ec-a43c7a704693 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:05:22.928013 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98389311-2fbe-4812-a7bc-02a7f68e778e admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:05:22.930662 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-98389311-2fbe-4812-a7bc-02a7f68e778e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:05:22.938562 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1098/2220] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:05:22 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:05:22.942441 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22661c5f-6139-4813-81ec-a43c7a704693 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:05:22.947482 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98389311-2fbe-4812-a7bc-02a7f68e778e admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:05:22.954111 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb41dd5-e2a0-4517-8bbf-8eaae34af54e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:05:22.955531 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb41dd5-e2a0-4517-8bbf-8eaae34af54e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:22.956136 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb41dd5-e2a0-4517-8bbf-8eaae34af54e admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:22.956420 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb41dd5-e2a0-4517-8bbf-8eaae34af54e admin admin] PATH_INFO: `/v3/projects/ef745aade6c44699b924d7badc1500f7` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:22.957719 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb41dd5-e2a0-4517-8bbf-8eaae34af54e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef745aade6c44699b924d7badc1500f7)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:05:22.959993 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb41dd5-e2a0-4517-8bbf-8eaae34af54e admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:05:22.966027 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22661c5f-6139-4813-81ec-a43c7a704693 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:05:22.967043 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d2d102-fd06-4fa6-b633-c1f82a20a5b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:05:22.967892 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d2d102-fd06-4fa6-b633-c1f82a20a5b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:22.968073 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d2d102-fd06-4fa6-b633-c1f82a20a5b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:22.968366 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d2d102-fd06-4fa6-b633-c1f82a20a5b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:22.969560 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d2d102-fd06-4fa6-b633-c1f82a20a5b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:05:22.971638 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d2d102-fd06-4fa6-b633-c1f82a20a5b9 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:05:22.993628 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1099/2221] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:05:22 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 10 13:05:22.994802 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d2d102-fd06-4fa6-b633-c1f82a20a5b9 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:05:23.005964 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d2d102-fd06-4fa6-b633-c1f82a20a5b9 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:05:23.010950 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d91d23-ed6e-4951-8f7e-713bf92f4e07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:05:23.011968 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d91d23-ed6e-4951-8f7e-713bf92f4e07 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:23.012402 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d91d23-ed6e-4951-8f7e-713bf92f4e07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:23.012689 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d91d23-ed6e-4951-8f7e-713bf92f4e07 admin admin] PATH_INFO: `/v3/projects/5beadd3890a3406cbef02fc3d4cd586b/users/7f0fc9ea47ce47649bfe84921103bd15/roles/43106c664e1b40de984c0c62701149c8` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:23.028242 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d91d23-ed6e-4951-8f7e-713bf92f4e07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5beadd3890a3406cbef02fc3d4cd586b, user_id=7f0fc9ea47ce47649bfe84921103bd15, role_id=43106c664e1b40de984c0c62701149c8)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:05:23.030868 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d91d23-ed6e-4951-8f7e-713bf92f4e07 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:05:23.103176 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1100/2222] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:05:22 2022] PUT /identity/v3/projects/5beadd3890a3406cbef02fc3d4cd586b/users/7f0fc9ea47ce47649bfe84921103bd15/roles/43106c664e1b40de984c0c62701149c8 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:05:23.113777 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ff94d7-9140-42b5-9402-9b8adb49bcef None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:23.114134 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ff94d7-9140-42b5-9402-9b8adb49bcef None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:23.114408 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ff94d7-9140-42b5-9402-9b8adb49bcef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:23.119707 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8eb41dd5-e2a0-4517-8bbf-8eaae34af54e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef745aade6c44699b924d7badc1500f7', 'request_id': 'req-8eb41dd5-e2a0-4517-8bbf-8eaae34af54e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:23.125320 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8eb41dd5-e2a0-4517-8bbf-8eaae34af54e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef745aade6c44699b924d7badc1500f7', 'request_id': 'req-8eb41dd5-e2a0-4517-8bbf-8eaae34af54e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:23.133626 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:05:23.133810 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8eb41dd5-e2a0-4517-8bbf-8eaae34af54e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef745aade6c44699b924d7badc1500f7', 'request_id': 'req-8eb41dd5-e2a0-4517-8bbf-8eaae34af54e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:23.152319 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-44ff94d7-9140-42b5-9402-9b8adb49bcef None None] MFA Rules not processed for user `7f0fc9ea47ce47649bfe84921103bd15`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:05:23.153514 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-44ff94d7-9140-42b5-9402-9b8adb49bcef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:05:23.222046 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1101/2223] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:05:23 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 10 13:05:23.224558 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8eb41dd5-e2a0-4517-8bbf-8eaae34af54e admin admin] The token cache is being invalidate because project ef745aade6c44699b924d7badc1500f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:05:23.224881 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8eb41dd5-e2a0-4517-8bbf-8eaae34af54e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:23.225552 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8eb41dd5-e2a0-4517-8bbf-8eaae34af54e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:23.227968 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1123/2224] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:05:22 2022] DELETE /identity/v3/projects/ef745aade6c44699b924d7badc1500f7 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:05:23.230480 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ff94d7-9140-42b5-9402-9b8adb49bcef None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:05:23.231365 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb41dd5-e2a0-4517-8bbf-8eaae34af54e admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:05:23.231901 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-44ff94d7-9140-42b5-9402-9b8adb49bcef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:05:23.232744 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8eb41dd5-e2a0-4517-8bbf-8eaae34af54e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:05:23.301762 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ff94d7-9140-42b5-9402-9b8adb49bcef None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:05:23.307357 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8e690e-df27-4ec1-8c30-3440969aebb2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6cc95a563b941ac92ab283aa0d5a5d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:05:23.308049 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8e690e-df27-4ec1-8c30-3440969aebb2 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:23.308332 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8e690e-df27-4ec1-8c30-3440969aebb2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:23.308608 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8e690e-df27-4ec1-8c30-3440969aebb2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:23.310783 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-ba8e690e-df27-4ec1-8c30-3440969aebb2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:05:23.327257 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb41dd5-e2a0-4517-8bbf-8eaae34af54e admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:05:23.327472 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8e690e-df27-4ec1-8c30-3440969aebb2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:05:23.329021 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8e690e-df27-4ec1-8c30-3440969aebb2 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:05:23.336032 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df0acc7-8fb5-4d12-9bad-61a72b0a6f25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:05:23.337081 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df0acc7-8fb5-4d12-9bad-61a72b0a6f25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:23.337441 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df0acc7-8fb5-4d12-9bad-61a72b0a6f25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:23.337798 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df0acc7-8fb5-4d12-9bad-61a72b0a6f25 admin admin] PATH_INFO: `/v3/users/74ac349d35d54fe3a278048c5cc8d37e` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:23.339333 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df0acc7-8fb5-4d12-9bad-61a72b0a6f25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=74ac349d35d54fe3a278048c5cc8d37e)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:05:23.340998 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1102/2225] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:05:23 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:05:23.341505 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df0acc7-8fb5-4d12-9bad-61a72b0a6f25 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:05:23.585842 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5df0acc7-8fb5-4d12-9bad-61a72b0a6f25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74ac349d35d54fe3a278048c5cc8d37e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:23.593017 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5df0acc7-8fb5-4d12-9bad-61a72b0a6f25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74ac349d35d54fe3a278048c5cc8d37e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:23.697695 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5df0acc7-8fb5-4d12-9bad-61a72b0a6f25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74ac349d35d54fe3a278048c5cc8d37e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:23.698233 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5df0acc7-8fb5-4d12-9bad-61a72b0a6f25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74ac349d35d54fe3a278048c5cc8d37e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:23.710608 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-5df0acc7-8fb5-4d12-9bad-61a72b0a6f25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74ac349d35d54fe3a278048c5cc8d37e'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:23.716916 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1124/2226] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:05:23 2022] DELETE /identity/v3/users/74ac349d35d54fe3a278048c5cc8d37e => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:05:23.731691 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:05:24.166301 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8e690e-df27-4ec1-8c30-3440969aebb2 service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:05:24.192684 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8e690e-df27-4ec1-8c30-3440969aebb2 service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:05:24.255093 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10746c6b-63bc-4556-81a5-75b4bf04ec78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:05:24.255372 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10746c6b-63bc-4556-81a5-75b4bf04ec78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:24.255688 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10746c6b-63bc-4556-81a5-75b4bf04ec78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:24.256078 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10746c6b-63bc-4556-81a5-75b4bf04ec78 admin admin] PATH_INFO: `/v3/projects/fd3f00b0e53f4216bef4673be036c44d` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:24.257375 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10746c6b-63bc-4556-81a5-75b4bf04ec78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd3f00b0e53f4216bef4673be036c44d)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:05:24.259947 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10746c6b-63bc-4556-81a5-75b4bf04ec78 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:05:24.402223 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:05:24.409296 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-10746c6b-63bc-4556-81a5-75b4bf04ec78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd3f00b0e53f4216bef4673be036c44d', 'request_id': 'req-10746c6b-63bc-4556-81a5-75b4bf04ec78'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:24.415112 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-10746c6b-63bc-4556-81a5-75b4bf04ec78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd3f00b0e53f4216bef4673be036c44d', 'request_id': 'req-10746c6b-63bc-4556-81a5-75b4bf04ec78'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:24.421683 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-10746c6b-63bc-4556-81a5-75b4bf04ec78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd3f00b0e53f4216bef4673be036c44d', 'request_id': 'req-10746c6b-63bc-4556-81a5-75b4bf04ec78'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:24.508702 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-10746c6b-63bc-4556-81a5-75b4bf04ec78 admin admin] The token cache is being invalidate because project fd3f00b0e53f4216bef4673be036c44d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:05:24.508907 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-10746c6b-63bc-4556-81a5-75b4bf04ec78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:24.509533 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-10746c6b-63bc-4556-81a5-75b4bf04ec78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:05:24.511081 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1103/2227] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:05:24 2022] DELETE /identity/v3/projects/fd3f00b0e53f4216bef4673be036c44d => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:05:24.515818 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b777cf5-d737-4f46-b506-ffd31cc64db0 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:24.516143 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b777cf5-d737-4f46-b506-ffd31cc64db0 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:24.516430 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b777cf5-d737-4f46-b506-ffd31cc64db0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:24.534091 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-8b777cf5-d737-4f46-b506-ffd31cc64db0 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:05:24.535322 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8b777cf5-d737-4f46-b506-ffd31cc64db0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:05:24.595380 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1125/2228] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:05:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:05:24.603783 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1a82b9-fd38-4e2c-b54a-c936407a492e None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:24.604152 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1a82b9-fd38-4e2c-b54a-c936407a492e None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:24.611282 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1a82b9-fd38-4e2c-b54a-c936407a492e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:24.623421 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-8e1a82b9-fd38-4e2c-b54a-c936407a492e None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:05:24.624615 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-8e1a82b9-fd38-4e2c-b54a-c936407a492e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:05:24.632801 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1104/2229] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:05:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 10 13:05:24.636657 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b777cf5-d737-4f46-b506-ffd31cc64db0 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:05:24.647617 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b777cf5-d737-4f46-b506-ffd31cc64db0 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:05:24.652528 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3facfdb3-f51b-4008-991c-7665ad2291a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:05:24.653193 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3facfdb3-f51b-4008-991c-7665ad2291a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:24.653477 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3facfdb3-f51b-4008-991c-7665ad2291a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:24.653758 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3facfdb3-f51b-4008-991c-7665ad2291a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:24.654491 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3facfdb3-f51b-4008-991c-7665ad2291a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 13:05:24.654794 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3facfdb3-f51b-4008-991c-7665ad2291a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:05:24.656467 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3facfdb3-f51b-4008-991c-7665ad2291a5 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:05:24.668708 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1126/2230] 10.222.0.27 () {58 vars in 1152 bytes} [Fri Jun 10 13:05:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 13:05:24.674517 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1a82b9-fd38-4e2c-b54a-c936407a492e None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:05:24.685486 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1a82b9-fd38-4e2c-b54a-c936407a492e None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:05:24.691774 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a70328-f4c9-46ea-94f1-d428f2d3e8ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:05:24.691774 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a70328-f4c9-46ea-94f1-d428f2d3e8ed admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:24.691774 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a70328-f4c9-46ea-94f1-d428f2d3e8ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:24.692145 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a70328-f4c9-46ea-94f1-d428f2d3e8ed admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:24.692803 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a70328-f4c9-46ea-94f1-d428f2d3e8ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:05:24.694763 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a70328-f4c9-46ea-94f1-d428f2d3e8ed admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:05:24.761726 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1105/2231] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:05:24 2022] POST /identity/v3/projects => generated 373 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:05:24.766009 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3facfdb3-f51b-4008-991c-7665ad2291a5 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:05:24.776644 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3facfdb3-f51b-4008-991c-7665ad2291a5 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:05:24.835212 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113e5b0c-9e54-462e-8a93-6475f9309958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:05:24.835926 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113e5b0c-9e54-462e-8a93-6475f9309958 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:24.836215 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113e5b0c-9e54-462e-8a93-6475f9309958 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:24.836491 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113e5b0c-9e54-462e-8a93-6475f9309958 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:24.837181 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113e5b0c-9e54-462e-8a93-6475f9309958 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:05:24.839233 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113e5b0c-9e54-462e-8a93-6475f9309958 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:05:24.916098 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:05:24.922200 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1127/2232] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:05:24 2022] POST /identity/v3/users => generated 287 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:05:24.926303 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a70328-f4c9-46ea-94f1-d428f2d3e8ed admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:05:24.937270 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a70328-f4c9-46ea-94f1-d428f2d3e8ed admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:05:24.942042 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c21284b-ecc8-4cef-a5ba-ec83c6395d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:05:24.942698 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c21284b-ecc8-4cef-a5ba-ec83c6395d59 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:24.942980 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c21284b-ecc8-4cef-a5ba-ec83c6395d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:24.943286 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c21284b-ecc8-4cef-a5ba-ec83c6395d59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:24.944041 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c21284b-ecc8-4cef-a5ba-ec83c6395d59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:05:24.945677 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c21284b-ecc8-4cef-a5ba-ec83c6395d59 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:05:24.956349 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1106/2233] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:05:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:05:24.960661 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113e5b0c-9e54-462e-8a93-6475f9309958 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:05:24.974139 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113e5b0c-9e54-462e-8a93-6475f9309958 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:05:24.978605 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e516f36-1622-4ac9-a63a-2d6e325f543f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:05:24.979441 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e516f36-1622-4ac9-a63a-2d6e325f543f admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:24.979724 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e516f36-1622-4ac9-a63a-2d6e325f543f admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:24.980042 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e516f36-1622-4ac9-a63a-2d6e325f543f admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:24.980781 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e516f36-1622-4ac9-a63a-2d6e325f543f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:05:24.983052 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e516f36-1622-4ac9-a63a-2d6e325f543f admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:05:24.994664 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1128/2234] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:05:24 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:05:24.999242 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c21284b-ecc8-4cef-a5ba-ec83c6395d59 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:05:25.009463 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c21284b-ecc8-4cef-a5ba-ec83c6395d59 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:05:25.014171 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c93b2d0-f5d3-474c-bd8c-1daaf3b7ca50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:05:25.014991 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c93b2d0-f5d3-474c-bd8c-1daaf3b7ca50 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:25.015272 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c93b2d0-f5d3-474c-bd8c-1daaf3b7ca50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:25.015546 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c93b2d0-f5d3-474c-bd8c-1daaf3b7ca50 admin admin] PATH_INFO: `/v3/projects/ba05afdeb3634c5f805e5eb87d0ffeba/users/83dd8f4c5eea426c9e027e470febec6a/roles/43106c664e1b40de984c0c62701149c8` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:25.034011 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c93b2d0-f5d3-474c-bd8c-1daaf3b7ca50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba05afdeb3634c5f805e5eb87d0ffeba, user_id=83dd8f4c5eea426c9e027e470febec6a, role_id=43106c664e1b40de984c0c62701149c8)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:05:25.036707 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c93b2d0-f5d3-474c-bd8c-1daaf3b7ca50 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:05:25.156273 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1107/2235] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:05:24 2022] PUT /identity/v3/projects/ba05afdeb3634c5f805e5eb87d0ffeba/users/83dd8f4c5eea426c9e027e470febec6a/roles/43106c664e1b40de984c0c62701149c8 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:05:25.167490 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a0fa15-41c9-457d-826c-acff4a0580b0 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:25.167805 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a0fa15-41c9-457d-826c-acff4a0580b0 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:25.168119 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a0fa15-41c9-457d-826c-acff4a0580b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:25.195207 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-b8a0fa15-41c9-457d-826c-acff4a0580b0 None None] MFA Rules not processed for user `83dd8f4c5eea426c9e027e470febec6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:05:25.197013 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b8a0fa15-41c9-457d-826c-acff4a0580b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:05:25.265780 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1129/2236] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:05:25 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:05:25.277434 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c93b2d0-f5d3-474c-bd8c-1daaf3b7ca50 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:05:25.279220 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0c93b2d0-f5d3-474c-bd8c-1daaf3b7ca50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:05:25.338530 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c93b2d0-f5d3-474c-bd8c-1daaf3b7ca50 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:05:25.342454 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee83f6c-3b25-46f1-9b5c-0b86b4956823 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6cc95a563b941ac92ab283aa0d5a5d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:05:25.343103 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee83f6c-3b25-46f1-9b5c-0b86b4956823 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:25.343388 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee83f6c-3b25-46f1-9b5c-0b86b4956823 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:25.343668 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee83f6c-3b25-46f1-9b5c-0b86b4956823 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:25.359226 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee83f6c-3b25-46f1-9b5c-0b86b4956823 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:05:25.360640 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee83f6c-3b25-46f1-9b5c-0b86b4956823 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:05:25.371933 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1108/2237] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:05:25 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 10 13:05:28.250573 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e82ff5-5b55-44e0-87e5-74adb95d8310 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:28.259069 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e82ff5-5b55-44e0-87e5-74adb95d8310 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:28.259069 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e82ff5-5b55-44e0-87e5-74adb95d8310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:28.269072 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-07e82ff5-5b55-44e0-87e5-74adb95d8310 None None] MFA Rules not processed for user `83dd8f4c5eea426c9e027e470febec6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:05:28.270354 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-07e82ff5-5b55-44e0-87e5-74adb95d8310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:05:28.277801 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1130/2238] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:05:28 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 10 13:05:35.367431 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=63,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:05:38.263574 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=41 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:05:55.410355 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26683809-d962-4bce-945c-f75421d895ab None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:55.410872 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26683809-d962-4bce-945c-f75421d895ab None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:55.410971 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26683809-d962-4bce-945c-f75421d895ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:55.481297 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-26683809-d962-4bce-945c-f75421d895ab None None] MFA Rules not processed for user `da66362070694e91bbe2cfbbd161b8a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:05:55.483025 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-26683809-d962-4bce-945c-f75421d895ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:05:55.555336 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1109/2239] 10.222.0.27 () {58 vars in 1009 bytes} [Fri Jun 10 13:05:55 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:05:55.595766 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e82ff5-5b55-44e0-87e5-74adb95d8310 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:05:55.597279 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-07e82ff5-5b55-44e0-87e5-74adb95d8310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:05:55.683735 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e82ff5-5b55-44e0-87e5-74adb95d8310 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:05:55.688645 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2255ef9b-5172-4350-a6f5-a53b48a6ac6a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb63b9cd74bf438aa25b72cef8337fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:05:55.689286 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2255ef9b-5172-4350-a6f5-a53b48a6ac6a service swift] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:55.689678 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2255ef9b-5172-4350-a6f5-a53b48a6ac6a service swift] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:55.689832 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2255ef9b-5172-4350-a6f5-a53b48a6ac6a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:55.698530 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2255ef9b-5172-4350-a6f5-a53b48a6ac6a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:05:55.700149 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2255ef9b-5172-4350-a6f5-a53b48a6ac6a service swift] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:05:55.708096 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1131/2240] 10.222.0.27 () {60 vars in 1384 bytes} [Fri Jun 10 13:05:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 10 13:05:56.554335 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74be458-140d-4b0b-b095-81b3d7609e98 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:56.554335 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74be458-140d-4b0b-b095-81b3d7609e98 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:56.554335 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74be458-140d-4b0b-b095-81b3d7609e98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:56.571064 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-d74be458-140d-4b0b-b095-81b3d7609e98 None None] MFA Rules not processed for user `da66362070694e91bbe2cfbbd161b8a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:05:56.572350 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d74be458-140d-4b0b-b095-81b3d7609e98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:05:56.580006 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1110/2241] 10.222.0.27 () {58 vars in 967 bytes} [Fri Jun 10 13:05:56 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:05:56.594345 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2255ef9b-5172-4350-a6f5-a53b48a6ac6a service swift] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:05:56.605607 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2255ef9b-5172-4350-a6f5-a53b48a6ac6a service swift] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:05:56.610933 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a9a566-81a3-4f76-80b5-42e7851907b4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb63b9cd74bf438aa25b72cef8337fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:05:56.611637 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a9a566-81a3-4f76-80b5-42e7851907b4 service swift] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:56.612036 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a9a566-81a3-4f76-80b5-42e7851907b4 service swift] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:56.612376 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a9a566-81a3-4f76-80b5-42e7851907b4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:56.620747 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a9a566-81a3-4f76-80b5-42e7851907b4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:05:56.622303 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a9a566-81a3-4f76-80b5-42e7851907b4 service swift] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:05:56.630196 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1132/2242] 10.222.0.27 () {60 vars in 1384 bytes} [Fri Jun 10 13:05:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 10 13:05:57.220642 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15febadb-c531-4813-84f6-6be459acadf6 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:57.220937 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15febadb-c531-4813-84f6-6be459acadf6 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:57.221222 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15febadb-c531-4813-84f6-6be459acadf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:57.238623 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-15febadb-c531-4813-84f6-6be459acadf6 None None] MFA Rules not processed for user `da66362070694e91bbe2cfbbd161b8a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:05:57.239816 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-15febadb-c531-4813-84f6-6be459acadf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:05:57.247445 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1111/2243] 10.222.0.27 () {58 vars in 1009 bytes} [Fri Jun 10 13:05:57 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:05:57.259525 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a9a566-81a3-4f76-80b5-42e7851907b4 service swift] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:05:57.269981 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a9a566-81a3-4f76-80b5-42e7851907b4 service swift] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:05:57.275182 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97487b37-0423-4458-a94b-e68f5726781f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb63b9cd74bf438aa25b72cef8337fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:05:57.275811 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97487b37-0423-4458-a94b-e68f5726781f service swift] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:57.276119 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97487b37-0423-4458-a94b-e68f5726781f service swift] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:57.276429 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97487b37-0423-4458-a94b-e68f5726781f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:57.284404 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97487b37-0423-4458-a94b-e68f5726781f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:05:57.285961 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97487b37-0423-4458-a94b-e68f5726781f service swift] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:05:57.293947 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1133/2244] 10.222.0.27 () {60 vars in 1384 bytes} [Fri Jun 10 13:05:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 10 13:05:57.846256 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe14513-b763-4dd4-8ba1-0d48a3d880c2 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:57.846715 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe14513-b763-4dd4-8ba1-0d48a3d880c2 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:57.846812 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe14513-b763-4dd4-8ba1-0d48a3d880c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:57.864809 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-bfe14513-b763-4dd4-8ba1-0d48a3d880c2 None None] MFA Rules not processed for user `da66362070694e91bbe2cfbbd161b8a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:05:57.866006 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-bfe14513-b763-4dd4-8ba1-0d48a3d880c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:05:57.873080 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1112/2245] 10.222.0.27 () {58 vars in 967 bytes} [Fri Jun 10 13:05:57 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:05:57.885401 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97487b37-0423-4458-a94b-e68f5726781f service swift] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:05:57.896293 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97487b37-0423-4458-a94b-e68f5726781f service swift] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:05:57.901962 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91acc1d9-464f-403a-968b-aaa6406aeccd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb63b9cd74bf438aa25b72cef8337fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:05:57.902624 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91acc1d9-464f-403a-968b-aaa6406aeccd service swift] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:05:57.902907 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91acc1d9-464f-403a-968b-aaa6406aeccd service swift] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:05:57.903186 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91acc1d9-464f-403a-968b-aaa6406aeccd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:05:57.911080 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91acc1d9-464f-403a-968b-aaa6406aeccd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:05:57.912679 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91acc1d9-464f-403a-968b-aaa6406aeccd service swift] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:05:57.921269 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1134/2246] 10.222.0.27 () {60 vars in 1384 bytes} [Fri Jun 10 13:05:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 10 13:06:00.037889 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe14513-b763-4dd4-8ba1-0d48a3d880c2 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:06:00.039807 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-bfe14513-b763-4dd4-8ba1-0d48a3d880c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:06:00.095658 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe14513-b763-4dd4-8ba1-0d48a3d880c2 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:06:00.099265 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9603ffe-4848-4fb5-81a0-6d63ed203f4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:06:00.099957 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9603ffe-4848-4fb5-81a0-6d63ed203f4a service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:06:00.100243 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9603ffe-4848-4fb5-81a0-6d63ed203f4a service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:06:00.100521 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9603ffe-4848-4fb5-81a0-6d63ed203f4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:06:00.102918 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f9603ffe-4848-4fb5-81a0-6d63ed203f4a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:06:00.150921 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:06:00.165142 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9603ffe-4848-4fb5-81a0-6d63ed203f4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:06:00.166587 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9603ffe-4848-4fb5-81a0-6d63ed203f4a service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:06:00.178011 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1113/2247] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:06:00 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:06:00.681872 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91acc1d9-464f-403a-968b-aaa6406aeccd service swift] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:06:00.747412 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91acc1d9-464f-403a-968b-aaa6406aeccd service swift] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:06:00.751949 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8251772d-262a-49d1-a886-056bfc38e29f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:06:00.752829 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8251772d-262a-49d1-a886-056bfc38e29f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:06:00.753107 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8251772d-262a-49d1-a886-056bfc38e29f admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:06:00.753383 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8251772d-262a-49d1-a886-056bfc38e29f admin admin] PATH_INFO: `/v3/users/83dd8f4c5eea426c9e027e470febec6a` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:06:00.754570 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8251772d-262a-49d1-a886-056bfc38e29f admin admin] RBAC: Authorizing `identity:delete_user(user_id=83dd8f4c5eea426c9e027e470febec6a)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:06:00.756506 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8251772d-262a-49d1-a886-056bfc38e29f admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:06:00.926217 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:06:00.938670 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8251772d-262a-49d1-a886-056bfc38e29f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83dd8f4c5eea426c9e027e470febec6a'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:06:00.945646 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8251772d-262a-49d1-a886-056bfc38e29f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83dd8f4c5eea426c9e027e470febec6a'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:06:01.003274 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8251772d-262a-49d1-a886-056bfc38e29f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83dd8f4c5eea426c9e027e470febec6a'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:06:01.003891 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8251772d-262a-49d1-a886-056bfc38e29f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83dd8f4c5eea426c9e027e470febec6a'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:06:01.016313 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8251772d-262a-49d1-a886-056bfc38e29f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83dd8f4c5eea426c9e027e470febec6a'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:06:01.023180 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1135/2248] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:06:00 2022] DELETE /identity/v3/users/83dd8f4c5eea426c9e027e470febec6a => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:06:01.027718 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b177ed4-8c04-41a1-a854-c3311281d697 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:06:01.028048 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b177ed4-8c04-41a1-a854-c3311281d697 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:06:01.028327 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b177ed4-8c04-41a1-a854-c3311281d697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:06:01.045477 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-3b177ed4-8c04-41a1-a854-c3311281d697 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:06:01.048330 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-3b177ed4-8c04-41a1-a854-c3311281d697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:06:01.110512 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1114/2249] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:06:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:06:01.120150 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8251772d-262a-49d1-a886-056bfc38e29f admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:06:01.121548 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8251772d-262a-49d1-a886-056bfc38e29f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:06:01.186980 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8251772d-262a-49d1-a886-056bfc38e29f admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:06:01.190609 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67504f8-5475-46ee-86cc-558c3e0f3ae7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:06:01.191339 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67504f8-5475-46ee-86cc-558c3e0f3ae7 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:06:01.191621 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67504f8-5475-46ee-86cc-558c3e0f3ae7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:06:01.191918 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67504f8-5475-46ee-86cc-558c3e0f3ae7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:06:01.201926 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67504f8-5475-46ee-86cc-558c3e0f3ae7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:06:01.203479 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67504f8-5475-46ee-86cc-558c3e0f3ae7 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:06:01.214378 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1136/2250] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:06:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:06:01.538591 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b177ed4-8c04-41a1-a854-c3311281d697 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:06:01.540091 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-3b177ed4-8c04-41a1-a854-c3311281d697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:06:01.551692 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b177ed4-8c04-41a1-a854-c3311281d697 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:06:01.556758 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a4cbfc-c91d-4262-a84f-e58cde268b74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:06:01.557639 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a4cbfc-c91d-4262-a84f-e58cde268b74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:06:01.557918 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a4cbfc-c91d-4262-a84f-e58cde268b74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:06:01.558189 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a4cbfc-c91d-4262-a84f-e58cde268b74 admin admin] PATH_INFO: `/v3/projects/ba05afdeb3634c5f805e5eb87d0ffeba` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:06:01.559373 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a4cbfc-c91d-4262-a84f-e58cde268b74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba05afdeb3634c5f805e5eb87d0ffeba)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:06:01.561306 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a4cbfc-c91d-4262-a84f-e58cde268b74 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:06:01.666075 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-e6a4cbfc-c91d-4262-a84f-e58cde268b74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba05afdeb3634c5f805e5eb87d0ffeba', 'request_id': 'req-e6a4cbfc-c91d-4262-a84f-e58cde268b74'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:06:01.671962 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-e6a4cbfc-c91d-4262-a84f-e58cde268b74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba05afdeb3634c5f805e5eb87d0ffeba', 'request_id': 'req-e6a4cbfc-c91d-4262-a84f-e58cde268b74'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:06:01.678271 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-e6a4cbfc-c91d-4262-a84f-e58cde268b74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba05afdeb3634c5f805e5eb87d0ffeba', 'request_id': 'req-e6a4cbfc-c91d-4262-a84f-e58cde268b74'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:06:01.762967 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-e6a4cbfc-c91d-4262-a84f-e58cde268b74 admin admin] The token cache is being invalidate because project ba05afdeb3634c5f805e5eb87d0ffeba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:06:01.763433 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-e6a4cbfc-c91d-4262-a84f-e58cde268b74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:06:01.764196 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-e6a4cbfc-c91d-4262-a84f-e58cde268b74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:06:01.765864 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1115/2251] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:06:01 2022] DELETE /identity/v3/projects/ba05afdeb3634c5f805e5eb87d0ffeba => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:06:10.675669 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67504f8-5475-46ee-86cc-558c3e0f3ae7 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:06:10.677175 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a67504f8-5475-46ee-86cc-558c3e0f3ae7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:06:10.690217 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67504f8-5475-46ee-86cc-558c3e0f3ae7 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:06:10.746296 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71254635-9406-4037-8257-ca5a0f93c1dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:06:10.747159 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71254635-9406-4037-8257-ca5a0f93c1dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:06:10.747445 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71254635-9406-4037-8257-ca5a0f93c1dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:06:10.747723 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71254635-9406-4037-8257-ca5a0f93c1dc admin admin] PATH_INFO: `/v3/users/7f0fc9ea47ce47649bfe84921103bd15` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:06:10.748921 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71254635-9406-4037-8257-ca5a0f93c1dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f0fc9ea47ce47649bfe84921103bd15)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:06:10.750813 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71254635-9406-4037-8257-ca5a0f93c1dc admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:06:10.920345 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-71254635-9406-4037-8257-ca5a0f93c1dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f0fc9ea47ce47649bfe84921103bd15'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:06:10.925611 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:06:10.929264 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-71254635-9406-4037-8257-ca5a0f93c1dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f0fc9ea47ce47649bfe84921103bd15'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:06:11.007987 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-71254635-9406-4037-8257-ca5a0f93c1dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f0fc9ea47ce47649bfe84921103bd15'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:06:11.008606 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-71254635-9406-4037-8257-ca5a0f93c1dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f0fc9ea47ce47649bfe84921103bd15'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:06:11.021305 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-71254635-9406-4037-8257-ca5a0f93c1dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f0fc9ea47ce47649bfe84921103bd15'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:06:11.027316 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1137/2252] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:06:10 2022] DELETE /identity/v3/users/7f0fc9ea47ce47649bfe84921103bd15 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:06:11.031801 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d661431e-e8a6-4cd2-861f-67221a1a1d93 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:06:11.032290 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d661431e-e8a6-4cd2-861f-67221a1a1d93 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:06:11.032475 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d661431e-e8a6-4cd2-861f-67221a1a1d93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:06:11.050585 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-d661431e-e8a6-4cd2-861f-67221a1a1d93 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:06:11.051774 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d661431e-e8a6-4cd2-861f-67221a1a1d93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:06:11.110735 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1116/2253] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:06:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:06:11.121006 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71254635-9406-4037-8257-ca5a0f93c1dc admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:06:11.122417 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-71254635-9406-4037-8257-ca5a0f93c1dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:06:11.184493 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71254635-9406-4037-8257-ca5a0f93c1dc admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:06:11.188113 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ebf6ff-55dc-4b42-a9ca-30262360897e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:06:11.188805 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ebf6ff-55dc-4b42-a9ca-30262360897e service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:06:11.189103 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ebf6ff-55dc-4b42-a9ca-30262360897e service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:06:11.189511 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ebf6ff-55dc-4b42-a9ca-30262360897e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:06:11.199743 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ebf6ff-55dc-4b42-a9ca-30262360897e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:06:11.201179 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ebf6ff-55dc-4b42-a9ca-30262360897e service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:06:11.210480 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1138/2254] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:06:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:06:11.643475 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d661431e-e8a6-4cd2-861f-67221a1a1d93 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:06:11.645025 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d661431e-e8a6-4cd2-861f-67221a1a1d93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:06:11.662015 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d661431e-e8a6-4cd2-861f-67221a1a1d93 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:06:11.667490 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb353d85-ae69-4ebc-9a71-71ae39052b2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:06:11.668390 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb353d85-ae69-4ebc-9a71-71ae39052b2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:06:11.668696 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb353d85-ae69-4ebc-9a71-71ae39052b2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:06:11.668976 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb353d85-ae69-4ebc-9a71-71ae39052b2f admin admin] PATH_INFO: `/v3/projects/5beadd3890a3406cbef02fc3d4cd586b` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:06:11.670513 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb353d85-ae69-4ebc-9a71-71ae39052b2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=5beadd3890a3406cbef02fc3d4cd586b)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:06:11.672657 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb353d85-ae69-4ebc-9a71-71ae39052b2f admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:06:11.680179 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:06:11.783720 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-bb353d85-ae69-4ebc-9a71-71ae39052b2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5beadd3890a3406cbef02fc3d4cd586b', 'request_id': 'req-bb353d85-ae69-4ebc-9a71-71ae39052b2f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:06:11.789118 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-bb353d85-ae69-4ebc-9a71-71ae39052b2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5beadd3890a3406cbef02fc3d4cd586b', 'request_id': 'req-bb353d85-ae69-4ebc-9a71-71ae39052b2f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:06:11.795793 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-bb353d85-ae69-4ebc-9a71-71ae39052b2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5beadd3890a3406cbef02fc3d4cd586b', 'request_id': 'req-bb353d85-ae69-4ebc-9a71-71ae39052b2f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:06:11.870251 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-bb353d85-ae69-4ebc-9a71-71ae39052b2f admin admin] The token cache is being invalidate because project 5beadd3890a3406cbef02fc3d4cd586b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:06:11.870536 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-bb353d85-ae69-4ebc-9a71-71ae39052b2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:06:11.871155 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-bb353d85-ae69-4ebc-9a71-71ae39052b2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:06:11.872746 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1117/2255] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:06:11 2022] DELETE /identity/v3/projects/5beadd3890a3406cbef02fc3d4cd586b => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:06:11.877476 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b2b729-632e-44c4-995b-5b47b1fbb42d None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:06:11.877876 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b2b729-632e-44c4-995b-5b47b1fbb42d None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:06:11.878029 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b2b729-632e-44c4-995b-5b47b1fbb42d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:06:11.896079 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-a7b2b729-632e-44c4-995b-5b47b1fbb42d None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:06:11.897325 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a7b2b729-632e-44c4-995b-5b47b1fbb42d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:06:11.954286 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1139/2256] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:06:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:06:11.963209 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9249a06-277c-445b-9400-abfebda678e0 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:06:11.963518 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9249a06-277c-445b-9400-abfebda678e0 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:06:11.963796 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9249a06-277c-445b-9400-abfebda678e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:06:11.980963 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-d9249a06-277c-445b-9400-abfebda678e0 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:06:11.982154 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d9249a06-277c-445b-9400-abfebda678e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:06:11.989988 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1118/2257] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:06:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:06:11.994534 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b2b729-632e-44c4-995b-5b47b1fbb42d None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:06:12.004186 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b2b729-632e-44c4-995b-5b47b1fbb42d None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:06:12.008734 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4581cdf4-b69b-4c7f-a4c0-af95cb797071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:06:12.009963 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4581cdf4-b69b-4c7f-a4c0-af95cb797071 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:06:12.010251 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4581cdf4-b69b-4c7f-a4c0-af95cb797071 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:06:12.010526 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4581cdf4-b69b-4c7f-a4c0-af95cb797071 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:06:12.011247 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4581cdf4-b69b-4c7f-a4c0-af95cb797071 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 13:06:12.011551 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4581cdf4-b69b-4c7f-a4c0-af95cb797071 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:06:12.013301 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4581cdf4-b69b-4c7f-a4c0-af95cb797071 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:06:12.024762 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1140/2258] 10.222.0.27 () {58 vars in 1152 bytes} [Fri Jun 10 13:06:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 13:06:12.028765 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9249a06-277c-445b-9400-abfebda678e0 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:06:12.038014 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9249a06-277c-445b-9400-abfebda678e0 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:06:12.042638 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe78e2d3-ca42-4b4f-a87c-d4ac06ab076c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:06:12.043288 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe78e2d3-ca42-4b4f-a87c-d4ac06ab076c admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:06:12.043573 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe78e2d3-ca42-4b4f-a87c-d4ac06ab076c admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:06:12.043853 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe78e2d3-ca42-4b4f-a87c-d4ac06ab076c admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:06:12.044571 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe78e2d3-ca42-4b4f-a87c-d4ac06ab076c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:06:12.046460 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe78e2d3-ca42-4b4f-a87c-d4ac06ab076c admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:06:12.149029 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1119/2259] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:06:12 2022] POST /identity/v3/projects => generated 373 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 10 13:06:12.153165 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4581cdf4-b69b-4c7f-a4c0-af95cb797071 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:06:12.163469 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4581cdf4-b69b-4c7f-a4c0-af95cb797071 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:06:12.220453 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d6d7bb-67a7-4635-bcbf-678b15281e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:06:12.221119 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d6d7bb-67a7-4635-bcbf-678b15281e01 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:06:12.221408 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d6d7bb-67a7-4635-bcbf-678b15281e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:06:12.221690 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d6d7bb-67a7-4635-bcbf-678b15281e01 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:06:12.222383 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d6d7bb-67a7-4635-bcbf-678b15281e01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:06:12.224501 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d6d7bb-67a7-4635-bcbf-678b15281e01 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:06:12.232035 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:06:12.367720 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1141/2260] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:06:12 2022] POST /identity/v3/users => generated 287 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:06:12.374908 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe78e2d3-ca42-4b4f-a87c-d4ac06ab076c admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:06:12.385627 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe78e2d3-ca42-4b4f-a87c-d4ac06ab076c admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:06:12.390031 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7484b687-e727-4b1c-9159-fabe6432ae40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:06:12.390679 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7484b687-e727-4b1c-9159-fabe6432ae40 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:06:12.390959 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7484b687-e727-4b1c-9159-fabe6432ae40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:06:12.391232 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7484b687-e727-4b1c-9159-fabe6432ae40 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:06:12.391991 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7484b687-e727-4b1c-9159-fabe6432ae40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:06:12.393569 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7484b687-e727-4b1c-9159-fabe6432ae40 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:06:12.403578 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1120/2261] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:06:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:06:12.407402 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d6d7bb-67a7-4635-bcbf-678b15281e01 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:06:12.418512 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d6d7bb-67a7-4635-bcbf-678b15281e01 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:06:12.424298 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafc604f-0a8e-444a-b0cd-c93cc4bbd5ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:06:12.424958 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafc604f-0a8e-444a-b0cd-c93cc4bbd5ac admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:06:12.425241 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafc604f-0a8e-444a-b0cd-c93cc4bbd5ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:06:12.425516 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafc604f-0a8e-444a-b0cd-c93cc4bbd5ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:06:12.426376 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafc604f-0a8e-444a-b0cd-c93cc4bbd5ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:06:12.427980 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafc604f-0a8e-444a-b0cd-c93cc4bbd5ac admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:06:12.439604 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1142/2262] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:06:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 10 13:06:12.443598 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7484b687-e727-4b1c-9159-fabe6432ae40 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:06:12.454642 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7484b687-e727-4b1c-9159-fabe6432ae40 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:06:12.460268 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a660af6-1480-4b2d-92f7-02cd81f8932e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:06:12.461180 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a660af6-1480-4b2d-92f7-02cd81f8932e admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:06:12.461455 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a660af6-1480-4b2d-92f7-02cd81f8932e admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:06:12.461725 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a660af6-1480-4b2d-92f7-02cd81f8932e admin admin] PATH_INFO: `/v3/projects/ff087154dbdb4b87af8c0daff640cb80/users/4d79a60e8fad4487b6ecd0044729acbd/roles/43106c664e1b40de984c0c62701149c8` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:06:12.476974 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a660af6-1480-4b2d-92f7-02cd81f8932e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff087154dbdb4b87af8c0daff640cb80, user_id=4d79a60e8fad4487b6ecd0044729acbd, role_id=43106c664e1b40de984c0c62701149c8)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:06:12.480148 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a660af6-1480-4b2d-92f7-02cd81f8932e admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:06:12.571163 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1121/2263] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:06:12 2022] PUT /identity/v3/projects/ff087154dbdb4b87af8c0daff640cb80/users/4d79a60e8fad4487b6ecd0044729acbd/roles/43106c664e1b40de984c0c62701149c8 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 10 13:06:12.581475 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e5e50b-56f4-477c-b0ad-a733b3cab1c8 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:06:12.581766 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e5e50b-56f4-477c-b0ad-a733b3cab1c8 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:06:12.582061 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e5e50b-56f4-477c-b0ad-a733b3cab1c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:06:12.609079 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-30e5e50b-56f4-477c-b0ad-a733b3cab1c8 None None] MFA Rules not processed for user `4d79a60e8fad4487b6ecd0044729acbd`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:06:12.610287 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-30e5e50b-56f4-477c-b0ad-a733b3cab1c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:06:12.679613 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1143/2264] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:06:12 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:06:12.691364 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a660af6-1480-4b2d-92f7-02cd81f8932e admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:06:12.693111 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5a660af6-1480-4b2d-92f7-02cd81f8932e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:06:12.772785 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a660af6-1480-4b2d-92f7-02cd81f8932e admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:06:12.777166 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5370e83f-845c-46f9-b962-5366958c63aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6cc95a563b941ac92ab283aa0d5a5d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:06:12.779052 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5370e83f-845c-46f9-b962-5366958c63aa service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:06:12.779052 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5370e83f-845c-46f9-b962-5366958c63aa service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:06:12.779052 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5370e83f-845c-46f9-b962-5366958c63aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:06:12.787016 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5370e83f-845c-46f9-b962-5366958c63aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:06:12.788496 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5370e83f-845c-46f9-b962-5366958c63aa service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:06:12.798258 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1122/2265] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:06:12 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 10 13:06:12.822923 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e5e50b-56f4-477c-b0ad-a733b3cab1c8 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:06:12.889819 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e5e50b-56f4-477c-b0ad-a733b3cab1c8 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:06:12.894975 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00259f80-4087-44d0-8abe-e3fc9930c366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:06:12.896191 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00259f80-4087-44d0-8abe-e3fc9930c366 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:06:12.896505 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00259f80-4087-44d0-8abe-e3fc9930c366 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:06:12.896781 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00259f80-4087-44d0-8abe-e3fc9930c366 admin admin] PATH_INFO: `/v3/users/4d79a60e8fad4487b6ecd0044729acbd` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:06:12.897985 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00259f80-4087-44d0-8abe-e3fc9930c366 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d79a60e8fad4487b6ecd0044729acbd)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:06:12.900552 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00259f80-4087-44d0-8abe-e3fc9930c366 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:06:13.125678 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-00259f80-4087-44d0-8abe-e3fc9930c366 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d79a60e8fad4487b6ecd0044729acbd'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:06:13.132465 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-00259f80-4087-44d0-8abe-e3fc9930c366 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d79a60e8fad4487b6ecd0044729acbd'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:06:13.198451 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-00259f80-4087-44d0-8abe-e3fc9930c366 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d79a60e8fad4487b6ecd0044729acbd'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:06:13.199041 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-00259f80-4087-44d0-8abe-e3fc9930c366 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d79a60e8fad4487b6ecd0044729acbd'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:06:13.206545 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:INSERT=3,keystone:SELECT=88,keystone:DELETE=9 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:06:13.211768 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-00259f80-4087-44d0-8abe-e3fc9930c366 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d79a60e8fad4487b6ecd0044729acbd'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:06:13.218276 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1144/2266] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:06:12 2022] DELETE /identity/v3/users/4d79a60e8fad4487b6ecd0044729acbd => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:06:13.222911 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e394c51-44b9-4bc6-a343-593b11fb5d8c None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:06:13.223383 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e394c51-44b9-4bc6-a343-593b11fb5d8c None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:06:13.223503 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e394c51-44b9-4bc6-a343-593b11fb5d8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:06:13.240883 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-3e394c51-44b9-4bc6-a343-593b11fb5d8c None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:06:13.242057 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-3e394c51-44b9-4bc6-a343-593b11fb5d8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:06:13.250205 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:06:13.299335 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1123/2267] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:06:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:06:13.308820 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00259f80-4087-44d0-8abe-e3fc9930c366 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:06:13.310187 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-00259f80-4087-44d0-8abe-e3fc9930c366 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:06:13.365536 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00259f80-4087-44d0-8abe-e3fc9930c366 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:06:13.369877 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74df5087-5263-48df-b575-f26bb85d2fd2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:06:13.370513 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74df5087-5263-48df-b575-f26bb85d2fd2 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:06:13.370794 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74df5087-5263-48df-b575-f26bb85d2fd2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:06:13.371068 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74df5087-5263-48df-b575-f26bb85d2fd2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:06:13.380127 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74df5087-5263-48df-b575-f26bb85d2fd2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:06:13.381535 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74df5087-5263-48df-b575-f26bb85d2fd2 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:06:13.391262 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1145/2268] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:06:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:06:13.800214 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e394c51-44b9-4bc6-a343-593b11fb5d8c None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:06:13.801634 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-3e394c51-44b9-4bc6-a343-593b11fb5d8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:06:13.814282 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e394c51-44b9-4bc6-a343-593b11fb5d8c None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:06:13.818967 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8baa39-f485-4407-b859-8c49e008189a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:06:13.819828 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8baa39-f485-4407-b859-8c49e008189a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:06:13.820141 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8baa39-f485-4407-b859-8c49e008189a admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:06:13.820408 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8baa39-f485-4407-b859-8c49e008189a admin admin] PATH_INFO: `/v3/projects/ff087154dbdb4b87af8c0daff640cb80` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:06:13.822182 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8baa39-f485-4407-b859-8c49e008189a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff087154dbdb4b87af8c0daff640cb80)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:06:13.824121 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8baa39-f485-4407-b859-8c49e008189a admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:06:13.947422 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-bc8baa39-f485-4407-b859-8c49e008189a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff087154dbdb4b87af8c0daff640cb80', 'request_id': 'req-bc8baa39-f485-4407-b859-8c49e008189a'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:06:13.953097 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-bc8baa39-f485-4407-b859-8c49e008189a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff087154dbdb4b87af8c0daff640cb80', 'request_id': 'req-bc8baa39-f485-4407-b859-8c49e008189a'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:06:13.959277 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-bc8baa39-f485-4407-b859-8c49e008189a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff087154dbdb4b87af8c0daff640cb80', 'request_id': 'req-bc8baa39-f485-4407-b859-8c49e008189a'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:06:14.037398 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-bc8baa39-f485-4407-b859-8c49e008189a admin admin] The token cache is being invalidate because project ff087154dbdb4b87af8c0daff640cb80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:06:14.037616 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-bc8baa39-f485-4407-b859-8c49e008189a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:06:14.038263 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-bc8baa39-f485-4407-b859-8c49e008189a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:06:14.040661 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1124/2269] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:06:13 2022] DELETE /identity/v3/projects/ff087154dbdb4b87af8c0daff640cb80 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:06:14.045540 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916b9956-5aa3-47fc-ad9e-12f70c06e132 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:06:14.046049 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916b9956-5aa3-47fc-ad9e-12f70c06e132 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:06:14.046155 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916b9956-5aa3-47fc-ad9e-12f70c06e132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:06:14.064085 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-916b9956-5aa3-47fc-ad9e-12f70c06e132 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:06:14.065279 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-916b9956-5aa3-47fc-ad9e-12f70c06e132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:06:14.122605 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1146/2270] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:06:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:06:14.130597 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17c393e-82c5-4c87-9487-acc0e60a2d7d None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:06:14.130891 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17c393e-82c5-4c87-9487-acc0e60a2d7d None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:06:14.131663 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17c393e-82c5-4c87-9487-acc0e60a2d7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:06:14.151778 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-d17c393e-82c5-4c87-9487-acc0e60a2d7d None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:06:14.153018 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d17c393e-82c5-4c87-9487-acc0e60a2d7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:06:14.161004 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1125/2271] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:06:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:06:14.165815 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916b9956-5aa3-47fc-ad9e-12f70c06e132 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:06:14.175407 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916b9956-5aa3-47fc-ad9e-12f70c06e132 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:06:14.180105 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5503e87d-15dd-4a00-85c2-1b97fb11d4de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:06:14.181423 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5503e87d-15dd-4a00-85c2-1b97fb11d4de admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:06:14.181706 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5503e87d-15dd-4a00-85c2-1b97fb11d4de admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:06:14.181980 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5503e87d-15dd-4a00-85c2-1b97fb11d4de admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:06:14.182706 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5503e87d-15dd-4a00-85c2-1b97fb11d4de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 13:06:14.183013 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5503e87d-15dd-4a00-85c2-1b97fb11d4de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:06:14.184649 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5503e87d-15dd-4a00-85c2-1b97fb11d4de admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:06:14.196209 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1147/2272] 10.222.0.27 () {58 vars in 1152 bytes} [Fri Jun 10 13:06:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 13:06:14.200496 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17c393e-82c5-4c87-9487-acc0e60a2d7d None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:06:14.209921 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17c393e-82c5-4c87-9487-acc0e60a2d7d None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:06:14.214707 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae318de9-5bfa-42ee-bad4-92d84316b482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:06:14.215381 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae318de9-5bfa-42ee-bad4-92d84316b482 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:06:14.215659 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae318de9-5bfa-42ee-bad4-92d84316b482 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:06:14.215960 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae318de9-5bfa-42ee-bad4-92d84316b482 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:06:14.216647 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae318de9-5bfa-42ee-bad4-92d84316b482 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:06:14.218786 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae318de9-5bfa-42ee-bad4-92d84316b482 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:06:14.330687 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1126/2273] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:06:14 2022] POST /identity/v3/projects => generated 385 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:06:14.334833 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5503e87d-15dd-4a00-85c2-1b97fb11d4de admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:06:14.345219 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5503e87d-15dd-4a00-85c2-1b97fb11d4de admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:06:14.401846 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaff9e06-2f30-4432-91e4-9e205c31971e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:06:14.402480 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaff9e06-2f30-4432-91e4-9e205c31971e admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:06:14.402764 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaff9e06-2f30-4432-91e4-9e205c31971e admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:06:14.403044 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaff9e06-2f30-4432-91e4-9e205c31971e admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:06:14.403725 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaff9e06-2f30-4432-91e4-9e205c31971e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:06:14.405657 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaff9e06-2f30-4432-91e4-9e205c31971e admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:06:14.524811 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1148/2274] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:06:14 2022] POST /identity/v3/users => generated 293 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:06:14.528824 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae318de9-5bfa-42ee-bad4-92d84316b482 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:06:14.539430 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae318de9-5bfa-42ee-bad4-92d84316b482 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:06:14.545391 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2c8d10-3f54-479a-9b90-25c672a879f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:06:14.546288 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2c8d10-3f54-479a-9b90-25c672a879f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:06:14.546661 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2c8d10-3f54-479a-9b90-25c672a879f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:06:14.547000 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2c8d10-3f54-479a-9b90-25c672a879f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:06:14.547823 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2c8d10-3f54-479a-9b90-25c672a879f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:06:14.549543 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2c8d10-3f54-479a-9b90-25c672a879f7 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:06:14.564518 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1127/2275] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:06:14 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:06:14.567238 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaff9e06-2f30-4432-91e4-9e205c31971e admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:06:14.577384 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaff9e06-2f30-4432-91e4-9e205c31971e admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:06:14.582463 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec72224-6b33-4b96-b0b8-ad6e47e219a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:06:14.583184 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec72224-6b33-4b96-b0b8-ad6e47e219a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:06:14.583468 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec72224-6b33-4b96-b0b8-ad6e47e219a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:06:14.583744 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec72224-6b33-4b96-b0b8-ad6e47e219a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:06:14.584516 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec72224-6b33-4b96-b0b8-ad6e47e219a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:06:14.586185 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec72224-6b33-4b96-b0b8-ad6e47e219a4 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:06:14.588931 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:06:14.597225 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1149/2276] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:06:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:06:14.601242 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2c8d10-3f54-479a-9b90-25c672a879f7 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:06:14.611943 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2c8d10-3f54-479a-9b90-25c672a879f7 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:06:14.616526 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de1ea69-8a29-4967-95bf-6eace2e32bd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:06:14.617399 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de1ea69-8a29-4967-95bf-6eace2e32bd7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:06:14.617746 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de1ea69-8a29-4967-95bf-6eace2e32bd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:06:14.618058 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de1ea69-8a29-4967-95bf-6eace2e32bd7 admin admin] PATH_INFO: `/v3/projects/7566f9559d464dcba426b5b404d4e29d/users/5195c19d6ce44b14b78565211d3ac234/roles/43106c664e1b40de984c0c62701149c8` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:06:14.632758 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de1ea69-8a29-4967-95bf-6eace2e32bd7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7566f9559d464dcba426b5b404d4e29d, user_id=5195c19d6ce44b14b78565211d3ac234, role_id=43106c664e1b40de984c0c62701149c8)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:06:14.640913 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de1ea69-8a29-4967-95bf-6eace2e32bd7 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:06:14.747065 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1128/2277] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:06:14 2022] PUT /identity/v3/projects/7566f9559d464dcba426b5b404d4e29d/users/5195c19d6ce44b14b78565211d3ac234/roles/43106c664e1b40de984c0c62701149c8 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:06:14.752127 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb8c9bf-5dc3-4a12-9a92-7358b04531e5 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:06:14.752421 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb8c9bf-5dc3-4a12-9a92-7358b04531e5 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:06:14.752707 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb8c9bf-5dc3-4a12-9a92-7358b04531e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:06:14.771817 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-6bb8c9bf-5dc3-4a12-9a92-7358b04531e5 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:06:14.773062 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6bb8c9bf-5dc3-4a12-9a92-7358b04531e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:06:14.830923 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1150/2278] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:06:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:06:14.840981 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de1ea69-8a29-4967-95bf-6eace2e32bd7 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:06:14.845701 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-7de1ea69-8a29-4967-95bf-6eace2e32bd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:06:14.885339 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:06:14.907837 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de1ea69-8a29-4967-95bf-6eace2e32bd7 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:06:14.911528 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fed07be-0ce6-43f3-8ed7-1b415b7d21b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:06:14.912286 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fed07be-0ce6-43f3-8ed7-1b415b7d21b2 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:06:14.912572 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fed07be-0ce6-43f3-8ed7-1b415b7d21b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:06:14.912851 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fed07be-0ce6-43f3-8ed7-1b415b7d21b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:06:14.922160 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fed07be-0ce6-43f3-8ed7-1b415b7d21b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:06:14.923686 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fed07be-0ce6-43f3-8ed7-1b415b7d21b2 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:06:14.933366 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1129/2279] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:06:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:06:20.815550 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d564eb-2a87-4d55-93f2-26b4d33edb3f None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:06:20.816082 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d564eb-2a87-4d55-93f2-26b4d33edb3f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:06:20.816189 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d564eb-2a87-4d55-93f2-26b4d33edb3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:06:20.848283 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-78d564eb-2a87-4d55-93f2-26b4d33edb3f None None] MFA Rules not processed for user `5195c19d6ce44b14b78565211d3ac234`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:06:20.850789 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-78d564eb-2a87-4d55-93f2-26b4d33edb3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:06:20.938371 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1151/2280] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:06:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:06:20.942832 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fed07be-0ce6-43f3-8ed7-1b415b7d21b2 service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:06:20.953525 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fed07be-0ce6-43f3-8ed7-1b415b7d21b2 service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:06:20.958550 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be42bbe4-d38c-400c-9516-9f93e732aca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:06:20.959419 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be42bbe4-d38c-400c-9516-9f93e732aca7 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:06:20.959603 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be42bbe4-d38c-400c-9516-9f93e732aca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:06:20.959909 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be42bbe4-d38c-400c-9516-9f93e732aca7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:06:20.960605 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be42bbe4-d38c-400c-9516-9f93e732aca7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:06:20.963042 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be42bbe4-d38c-400c-9516-9f93e732aca7 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:06:21.079727 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1130/2281] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:06:20 2022] POST /identity/v3/projects => generated 385 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:06:21.083775 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d564eb-2a87-4d55-93f2-26b4d33edb3f None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:06:21.097168 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d564eb-2a87-4d55-93f2-26b4d33edb3f None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:06:21.159090 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d757467-1b26-43ad-aac9-ed42cf66c85a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:06:21.160076 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d757467-1b26-43ad-aac9-ed42cf66c85a admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:06:21.160612 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d757467-1b26-43ad-aac9-ed42cf66c85a admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:06:21.161057 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d757467-1b26-43ad-aac9-ed42cf66c85a admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:06:21.161988 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d757467-1b26-43ad-aac9-ed42cf66c85a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:06:21.164134 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d757467-1b26-43ad-aac9-ed42cf66c85a admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:06:21.277345 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1152/2282] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:06:21 2022] POST /identity/v3/users => generated 299 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:06:21.281484 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be42bbe4-d38c-400c-9516-9f93e732aca7 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:06:21.292877 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be42bbe4-d38c-400c-9516-9f93e732aca7 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:06:21.297756 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b0bd10-665a-4ac5-85e4-ed48427e5e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:06:21.298441 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b0bd10-665a-4ac5-85e4-ed48427e5e01 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:06:21.298727 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b0bd10-665a-4ac5-85e4-ed48427e5e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:06:21.299006 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b0bd10-665a-4ac5-85e4-ed48427e5e01 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:06:21.299746 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b0bd10-665a-4ac5-85e4-ed48427e5e01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:06:21.301391 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b0bd10-665a-4ac5-85e4-ed48427e5e01 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:06:21.312566 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1131/2283] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:06:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 10 13:06:21.316666 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d757467-1b26-43ad-aac9-ed42cf66c85a admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:06:21.326696 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d757467-1b26-43ad-aac9-ed42cf66c85a admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:06:21.331282 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf98a3f-6813-40c1-ab88-c225e1946499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:06:21.332186 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf98a3f-6813-40c1-ab88-c225e1946499 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:06:21.332477 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf98a3f-6813-40c1-ab88-c225e1946499 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:06:21.332755 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf98a3f-6813-40c1-ab88-c225e1946499 admin admin] PATH_INFO: `/v3/projects/cbdee25e120247ac9be33bc68d2e82b3/users/0321863f96a142ad8585602ddfa86946/roles/b43d5a8247e244869c2c70f986f42e8a` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:06:21.347665 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf98a3f-6813-40c1-ab88-c225e1946499 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbdee25e120247ac9be33bc68d2e82b3, user_id=0321863f96a142ad8585602ddfa86946, role_id=b43d5a8247e244869c2c70f986f42e8a)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:06:21.350321 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf98a3f-6813-40c1-ab88-c225e1946499 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:06:21.352582 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:06:21.460874 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1153/2284] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:06:21 2022] PUT /identity/v3/projects/cbdee25e120247ac9be33bc68d2e82b3/users/0321863f96a142ad8585602ddfa86946/roles/b43d5a8247e244869c2c70f986f42e8a => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 10 13:06:21.465568 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b0bd10-665a-4ac5-85e4-ed48427e5e01 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:06:21.475893 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b0bd10-665a-4ac5-85e4-ed48427e5e01 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:06:21.535167 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dec7b2-4df6-4163-9661-369215642baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:06:21.535762 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dec7b2-4df6-4163-9661-369215642baa admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:06:21.536074 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dec7b2-4df6-4163-9661-369215642baa admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:06:21.536350 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dec7b2-4df6-4163-9661-369215642baa admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:06:21.537068 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dec7b2-4df6-4163-9661-369215642baa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:06:21.538661 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dec7b2-4df6-4163-9661-369215642baa admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:06:21.548114 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f68eef-8fba-4487-ab98-bdec1f2d3e4b None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:06:21.549629 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f68eef-8fba-4487-ab98-bdec1f2d3e4b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:06:21.549629 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f68eef-8fba-4487-ab98-bdec1f2d3e4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:06:21.549830 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1132/2285] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:06:21 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:06:21.553107 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dec7b2-4df6-4163-9661-369215642baa admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:06:21.562891 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dec7b2-4df6-4163-9661-369215642baa admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:06:21.567505 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-05f68eef-8fba-4487-ab98-bdec1f2d3e4b None None] MFA Rules not processed for user `8e32dbeb85bd4fb889d6dde932c60853`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:06:21.567782 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7a62a4-2cbb-4bba-94f7-6ed846e101c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:06:21.568709 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7a62a4-2cbb-4bba-94f7-6ed846e101c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:06:21.568848 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-05f68eef-8fba-4487-ab98-bdec1f2d3e4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:06:21.568993 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7a62a4-2cbb-4bba-94f7-6ed846e101c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:06:21.569267 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7a62a4-2cbb-4bba-94f7-6ed846e101c3 admin admin] PATH_INFO: `/v3/projects/cbdee25e120247ac9be33bc68d2e82b3/users/0321863f96a142ad8585602ddfa86946/roles/b43d5a8247e244869c2c70f986f42e8a` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:06:21.571261 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7a62a4-2cbb-4bba-94f7-6ed846e101c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbdee25e120247ac9be33bc68d2e82b3, user_id=0321863f96a142ad8585602ddfa86946, role_id=b43d5a8247e244869c2c70f986f42e8a)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:06:21.578333 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7a62a4-2cbb-4bba-94f7-6ed846e101c3 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:06:21.586477 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1133/2286] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:06:21 2022] PUT /identity/v3/projects/cbdee25e120247ac9be33bc68d2e82b3/users/0321863f96a142ad8585602ddfa86946/roles/b43d5a8247e244869c2c70f986f42e8a => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 10 13:06:21.630249 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1154/2287] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:06:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 10 13:06:24.050150 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7a62a4-2cbb-4bba-94f7-6ed846e101c3 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:06:24.052021 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-2d7a62a4-2cbb-4bba-94f7-6ed846e101c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:06:24.175081 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7a62a4-2cbb-4bba-94f7-6ed846e101c3 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:06:24.180196 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d37116-4a65-44e6-870e-458993b602d1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd972ab782b3f4d2aba575314f16dafc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:06:24.180844 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d37116-4a65-44e6-870e-458993b602d1 service placement] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:06:24.181662 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d37116-4a65-44e6-870e-458993b602d1 service placement] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:06:24.181944 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d37116-4a65-44e6-870e-458993b602d1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:06:24.184063 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a7d37116-4a65-44e6-870e-458993b602d1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:06:24.194418 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:06:24.197665 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d37116-4a65-44e6-870e-458993b602d1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:06:24.199763 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d37116-4a65-44e6-870e-458993b602d1 service placement] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:06:24.212435 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1134/2288] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:06:24 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:06:29.160383 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7e922d-cbb2-44e8-99b1-184ac4045c08 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:06:29.160383 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7e922d-cbb2-44e8-99b1-184ac4045c08 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:06:29.160383 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7e922d-cbb2-44e8-99b1-184ac4045c08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:06:29.185780 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-8d7e922d-cbb2-44e8-99b1-184ac4045c08 None None] MFA Rules not processed for user `0321863f96a142ad8585602ddfa86946`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:06:29.187113 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8d7e922d-cbb2-44e8-99b1-184ac4045c08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:06:29.277622 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1155/2289] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:06:29 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:06:29.290239 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d37116-4a65-44e6-870e-458993b602d1 service placement] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:06:29.292230 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a7d37116-4a65-44e6-870e-458993b602d1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:06:29.392946 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d37116-4a65-44e6-870e-458993b602d1 service placement] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:06:29.397937 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7567f7-91a9-49ef-9cec-5e7633e06a9f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d3eb01a98274050b930616b98998c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:06:29.398703 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7567f7-91a9-49ef-9cec-5e7633e06a9f service glance] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:06:29.399060 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7567f7-91a9-49ef-9cec-5e7633e06a9f service glance] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:06:29.399568 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7567f7-91a9-49ef-9cec-5e7633e06a9f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:06:29.475898 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7567f7-91a9-49ef-9cec-5e7633e06a9f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:06:29.477679 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7567f7-91a9-49ef-9cec-5e7633e06a9f service glance] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:06:29.488466 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1135/2290] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:06:29 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 10 13:06:29.746208 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7382e9-dd3f-45a8-8fe5-cd506afd4375 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:06:29.746503 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7382e9-dd3f-45a8-8fe5-cd506afd4375 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:06:29.746785 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7382e9-dd3f-45a8-8fe5-cd506afd4375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:06:29.764667 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-ca7382e9-dd3f-45a8-8fe5-cd506afd4375 None None] MFA Rules not processed for user `da66362070694e91bbe2cfbbd161b8a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:06:29.765512 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ca7382e9-dd3f-45a8-8fe5-cd506afd4375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:06:29.832744 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1156/2291] 10.222.0.27 () {58 vars in 1009 bytes} [Fri Jun 10 13:06:29 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:06:29.846914 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7567f7-91a9-49ef-9cec-5e7633e06a9f service glance] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:06:29.848662 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0a7567f7-91a9-49ef-9cec-5e7633e06a9f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:06:29.921536 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7567f7-91a9-49ef-9cec-5e7633e06a9f service glance] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:06:29.926649 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100cb2ae-8e34-4179-8500-d7be6e892ee6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb63b9cd74bf438aa25b72cef8337fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:06:29.927319 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100cb2ae-8e34-4179-8500-d7be6e892ee6 service swift] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:06:29.927607 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100cb2ae-8e34-4179-8500-d7be6e892ee6 service swift] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:06:29.927908 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100cb2ae-8e34-4179-8500-d7be6e892ee6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:06:29.937641 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100cb2ae-8e34-4179-8500-d7be6e892ee6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:06:29.939371 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100cb2ae-8e34-4179-8500-d7be6e892ee6 service swift] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:06:29.948606 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1136/2292] 10.222.0.27 () {60 vars in 1384 bytes} [Fri Jun 10 13:06:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 10 13:06:35.887831 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee472af-490c-46b2-816e-383e91287d74 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:06:35.888384 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee472af-490c-46b2-816e-383e91287d74 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:06:35.888484 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee472af-490c-46b2-816e-383e91287d74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:06:35.908805 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-7ee472af-490c-46b2-816e-383e91287d74 None None] MFA Rules not processed for user `da66362070694e91bbe2cfbbd161b8a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:06:35.910406 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-7ee472af-490c-46b2-816e-383e91287d74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:06:35.919059 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1157/2293] 10.222.0.27 () {58 vars in 1009 bytes} [Fri Jun 10 13:06:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:06:35.935554 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100cb2ae-8e34-4179-8500-d7be6e892ee6 service swift] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:06:35.947104 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100cb2ae-8e34-4179-8500-d7be6e892ee6 service swift] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:06:35.953001 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaf8958-44e3-44e0-b3ed-d781e705c050 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb63b9cd74bf438aa25b72cef8337fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:06:35.953930 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaf8958-44e3-44e0-b3ed-d781e705c050 service swift] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:06:35.954369 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaf8958-44e3-44e0-b3ed-d781e705c050 service swift] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:06:35.954802 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaf8958-44e3-44e0-b3ed-d781e705c050 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:06:35.965373 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaf8958-44e3-44e0-b3ed-d781e705c050 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:06:35.967185 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaf8958-44e3-44e0-b3ed-d781e705c050 service swift] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:06:35.978269 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1137/2294] 10.222.0.27 () {60 vars in 1384 bytes} [Fri Jun 10 13:06:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 10 13:06:43.283835 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9505f093-79b5-4af2-9cdd-869f6b3e10a6 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:06:43.284462 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9505f093-79b5-4af2-9cdd-869f6b3e10a6 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:06:43.284656 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9505f093-79b5-4af2-9cdd-869f6b3e10a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:06:43.308628 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-9505f093-79b5-4af2-9cdd-869f6b3e10a6 None None] MFA Rules not processed for user `5195c19d6ce44b14b78565211d3ac234`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:06:43.310011 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9505f093-79b5-4af2-9cdd-869f6b3e10a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:06:43.323817 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1158/2295] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:06:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:06:45.973705 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:06:47.706478 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af14c9a-dc4a-4b92-8faf-fdcc90deb670 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:06:47.707161 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af14c9a-dc4a-4b92-8faf-fdcc90deb670 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:06:47.707161 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af14c9a-dc4a-4b92-8faf-fdcc90deb670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:06:47.737466 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-8af14c9a-dc4a-4b92-8faf-fdcc90deb670 None None] MFA Rules not processed for user `da66362070694e91bbe2cfbbd161b8a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:06:47.738858 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-8af14c9a-dc4a-4b92-8faf-fdcc90deb670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:06:47.760439 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1138/2296] 10.222.0.27 () {58 vars in 1009 bytes} [Fri Jun 10 13:06:47 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:06:47.779190 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9505f093-79b5-4af2-9cdd-869f6b3e10a6 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:06:47.791708 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9505f093-79b5-4af2-9cdd-869f6b3e10a6 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:06:47.806104 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81175f2a-aa7e-4741-bc80-4668e611c147 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb63b9cd74bf438aa25b72cef8337fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:06:47.807024 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81175f2a-aa7e-4741-bc80-4668e611c147 service swift] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:06:47.807491 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81175f2a-aa7e-4741-bc80-4668e611c147 service swift] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:06:47.808043 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81175f2a-aa7e-4741-bc80-4668e611c147 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:06:47.820253 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81175f2a-aa7e-4741-bc80-4668e611c147 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:06:47.822755 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81175f2a-aa7e-4741-bc80-4668e611c147 service swift] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:06:47.833070 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1159/2297] 10.222.0.27 () {60 vars in 1384 bytes} [Fri Jun 10 13:06:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 10 13:06:57.731544 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:06:57.828512 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:INSERT=1,keystone:SELECT=98 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:07:12.543773 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622f03ce-ad42-4d80-adca-95fd388d106d None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:07:12.544835 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622f03ce-ad42-4d80-adca-95fd388d106d None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:07:12.545184 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622f03ce-ad42-4d80-adca-95fd388d106d None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:07:12.547380 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1139/2298] 10.222.0.27 () {54 vars in 921 bytes} [Fri Jun 10 13:07:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 13:07:12.553547 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc91ed8b-294c-4531-a8ba-f2ab9389fa5b None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:07:12.554065 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc91ed8b-294c-4531-a8ba-f2ab9389fa5b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:07:12.554226 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc91ed8b-294c-4531-a8ba-f2ab9389fa5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:07:12.608566 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-fc91ed8b-294c-4531-a8ba-f2ab9389fa5b None None] MFA Rules not processed for user `8bd6caa461e84100a3d5fe5af5d80f3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:07:12.609787 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-fc91ed8b-294c-4531-a8ba-f2ab9389fa5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:07:12.685626 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1160/2299] 10.222.0.27 () {58 vars in 1018 bytes} [Fri Jun 10 13:07:12 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:07:12.697385 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622f03ce-ad42-4d80-adca-95fd388d106d None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:07:12.709360 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622f03ce-ad42-4d80-adca-95fd388d106d None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:07:12.713147 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fd034f-e567-44b5-9d6d-b727a27e989e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d3eb01a98274050b930616b98998c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:07:12.714470 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fd034f-e567-44b5-9d6d-b727a27e989e service glance] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:07:12.714761 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fd034f-e567-44b5-9d6d-b727a27e989e service glance] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:07:12.715043 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fd034f-e567-44b5-9d6d-b727a27e989e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:07:12.730861 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fd034f-e567-44b5-9d6d-b727a27e989e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:07:12.732429 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fd034f-e567-44b5-9d6d-b727a27e989e service glance] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:07:12.744016 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1140/2300] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:07:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 10 13:07:12.926191 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b992b02-a5ff-403f-bca6-fec57f50b0df None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:07:12.926488 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b992b02-a5ff-403f-bca6-fec57f50b0df None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:07:12.926848 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b992b02-a5ff-403f-bca6-fec57f50b0df None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:07:12.928271 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1161/2301] 10.222.0.27 () {54 vars in 921 bytes} [Fri Jun 10 13:07:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 13:07:12.933703 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1863ef1e-4887-4a05-af4a-325d157083f8 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:07:12.934059 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1863ef1e-4887-4a05-af4a-325d157083f8 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:07:12.934269 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1863ef1e-4887-4a05-af4a-325d157083f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:07:12.956641 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-1863ef1e-4887-4a05-af4a-325d157083f8 None None] MFA Rules not processed for user `bc4415461a974c24ad7105a97c7d915b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:07:12.956641 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-1863ef1e-4887-4a05-af4a-325d157083f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:07:13.047641 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1141/2302] 10.222.0.27 () {58 vars in 1018 bytes} [Fri Jun 10 13:07:12 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 10 13:07:13.067581 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b992b02-a5ff-403f-bca6-fec57f50b0df None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:07:13.069372 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2b992b02-a5ff-403f-bca6-fec57f50b0df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:07:13.086976 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b992b02-a5ff-403f-bca6-fec57f50b0df None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:07:13.095086 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d163eb0-2176-4b19-aac9-368820aab277 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:07:13.107298 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d163eb0-2176-4b19-aac9-368820aab277 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:07:13.107658 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d163eb0-2176-4b19-aac9-368820aab277 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:07:13.107973 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d163eb0-2176-4b19-aac9-368820aab277 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:07:13.126672 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d163eb0-2176-4b19-aac9-368820aab277 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:07:13.131940 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d163eb0-2176-4b19-aac9-368820aab277 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:07:13.141907 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1162/2303] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:07:13 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:07:13.231158 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc07d160-4be5-4a2f-be52-e0fefeee74ac None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:07:13.231451 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc07d160-4be5-4a2f-be52-e0fefeee74ac None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:07:13.231732 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc07d160-4be5-4a2f-be52-e0fefeee74ac None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:07:13.233148 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1142/2304] 10.222.0.27 () {54 vars in 921 bytes} [Fri Jun 10 13:07:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 13:07:13.238491 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbd4b8f-3432-4618-9ee0-8951decffb29 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:07:13.238782 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbd4b8f-3432-4618-9ee0-8951decffb29 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:07:13.239067 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbd4b8f-3432-4618-9ee0-8951decffb29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:07:13.257154 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-cbbd4b8f-3432-4618-9ee0-8951decffb29 None None] MFA Rules not processed for user `8bd6caa461e84100a3d5fe5af5d80f3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:07:13.258358 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-cbbd4b8f-3432-4618-9ee0-8951decffb29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:07:13.268245 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1163/2305] 10.222.0.27 () {58 vars in 1018 bytes} [Fri Jun 10 13:07:13 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:07:13.279831 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc07d160-4be5-4a2f-be52-e0fefeee74ac None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:07:13.290494 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc07d160-4be5-4a2f-be52-e0fefeee74ac None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:07:13.294004 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda463f3-4d06-4e04-985c-f477cee1c456 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:07:13.294629 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda463f3-4d06-4e04-985c-f477cee1c456 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:07:13.294910 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda463f3-4d06-4e04-985c-f477cee1c456 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:07:13.295185 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda463f3-4d06-4e04-985c-f477cee1c456 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:07:13.305141 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda463f3-4d06-4e04-985c-f477cee1c456 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:07:13.306551 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda463f3-4d06-4e04-985c-f477cee1c456 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:07:13.317060 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1143/2306] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:07:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:07:18.541399 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef75f013-736c-4a4f-bc6d-d3abc9d68cf8 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:07:18.552500 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef75f013-736c-4a4f-bc6d-d3abc9d68cf8 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:07:18.552500 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef75f013-736c-4a4f-bc6d-d3abc9d68cf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:07:18.560859 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-ef75f013-736c-4a4f-bc6d-d3abc9d68cf8 None None] MFA Rules not processed for user `da66362070694e91bbe2cfbbd161b8a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:07:18.562044 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ef75f013-736c-4a4f-bc6d-d3abc9d68cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:07:18.570832 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1164/2307] 10.222.0.27 () {58 vars in 1009 bytes} [Fri Jun 10 13:07:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:07:18.594036 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda463f3-4d06-4e04-985c-f477cee1c456 service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:07:18.605406 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda463f3-4d06-4e04-985c-f477cee1c456 service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:07:18.610418 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089a8429-e9f2-4db4-bf83-84c758d873e0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb63b9cd74bf438aa25b72cef8337fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:07:18.611057 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089a8429-e9f2-4db4-bf83-84c758d873e0 service swift] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:07:18.611370 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089a8429-e9f2-4db4-bf83-84c758d873e0 service swift] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:07:18.611653 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089a8429-e9f2-4db4-bf83-84c758d873e0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:07:18.622003 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089a8429-e9f2-4db4-bf83-84c758d873e0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:07:18.623732 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089a8429-e9f2-4db4-bf83-84c758d873e0 service swift] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:07:18.632758 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1144/2308] 10.222.0.27 () {60 vars in 1384 bytes} [Fri Jun 10 13:07:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 10 13:07:28.555050 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=58 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:07:28.629053 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=41 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:07:42.066488 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef75f013-736c-4a4f-bc6d-d3abc9d68cf8 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:07:42.076507 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef75f013-736c-4a4f-bc6d-d3abc9d68cf8 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:07:42.080343 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b811f6e1-7fe4-410d-8f6d-d62cfcdb54bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:07:42.081021 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b811f6e1-7fe4-410d-8f6d-d62cfcdb54bd service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:07:42.081304 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b811f6e1-7fe4-410d-8f6d-d62cfcdb54bd service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:07:42.081581 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b811f6e1-7fe4-410d-8f6d-d62cfcdb54bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:07:42.091101 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b811f6e1-7fe4-410d-8f6d-d62cfcdb54bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:07:42.100844 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b811f6e1-7fe4-410d-8f6d-d62cfcdb54bd service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:07:42.101997 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1165/2309] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:07:42 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:07:45.997590 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c287815f-9a61-4ec9-9c8c-a04eb1d91e4a None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:07:45.998147 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c287815f-9a61-4ec9-9c8c-a04eb1d91e4a None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:07:45.998147 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c287815f-9a61-4ec9-9c8c-a04eb1d91e4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:07:46.023801 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-c287815f-9a61-4ec9-9c8c-a04eb1d91e4a None None] MFA Rules not processed for user `da66362070694e91bbe2cfbbd161b8a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:07:46.025053 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c287815f-9a61-4ec9-9c8c-a04eb1d91e4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:07:46.033022 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1145/2310] 10.222.0.27 () {58 vars in 967 bytes} [Fri Jun 10 13:07:45 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 10 13:07:46.046381 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b811f6e1-7fe4-410d-8f6d-d62cfcdb54bd service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:07:46.058041 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b811f6e1-7fe4-410d-8f6d-d62cfcdb54bd service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:07:46.063457 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2f12a5-f084-477a-8849-00945d874433 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb63b9cd74bf438aa25b72cef8337fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:07:46.064140 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2f12a5-f084-477a-8849-00945d874433 service swift] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:07:46.064428 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2f12a5-f084-477a-8849-00945d874433 service swift] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:07:46.064709 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2f12a5-f084-477a-8849-00945d874433 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:07:46.073891 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2f12a5-f084-477a-8849-00945d874433 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:07:46.075888 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2f12a5-f084-477a-8849-00945d874433 service swift] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:07:46.085246 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1166/2311] 10.222.0.27 () {60 vars in 1384 bytes} [Fri Jun 10 13:07:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 10 13:07:56.017648 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:07:56.081546 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=12 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:08:06.952529 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d434f04f-bcea-40e0-9cbd-b345bd437d17 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:08:06.953092 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d434f04f-bcea-40e0-9cbd-b345bd437d17 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:08:06.953092 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d434f04f-bcea-40e0-9cbd-b345bd437d17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:08:06.970179 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-d434f04f-bcea-40e0-9cbd-b345bd437d17 None None] MFA Rules not processed for user `e0b63da5bf0b4410acc9ff573064f8ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:08:06.971391 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d434f04f-bcea-40e0-9cbd-b345bd437d17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:08:07.028113 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1146/2312] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:08:06 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:08:17.020938 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=25 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:08:20.810767 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2f12a5-f084-477a-8849-00945d874433 service swift] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:08:20.820048 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2f12a5-f084-477a-8849-00945d874433 service swift] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:08:20.823546 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdddae5e-6bb9-4026-a78b-dd3cba3a392e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:08:20.824716 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdddae5e-6bb9-4026-a78b-dd3cba3a392e service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:08:20.824997 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdddae5e-6bb9-4026-a78b-dd3cba3a392e service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:08:20.825272 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdddae5e-6bb9-4026-a78b-dd3cba3a392e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:08:20.827027 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-fdddae5e-6bb9-4026-a78b-dd3cba3a392e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:08:20.840363 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdddae5e-6bb9-4026-a78b-dd3cba3a392e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:08:20.841799 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdddae5e-6bb9-4026-a78b-dd3cba3a392e service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:08:20.852360 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1167/2313] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:08:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:08:22.019146 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d434f04f-bcea-40e0-9cbd-b345bd437d17 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:08:22.029375 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d434f04f-bcea-40e0-9cbd-b345bd437d17 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:08:22.032991 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f350fd66-05dc-4069-8975-0da579d09d97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:08:22.033633 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f350fd66-05dc-4069-8975-0da579d09d97 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:08:22.033950 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f350fd66-05dc-4069-8975-0da579d09d97 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:08:22.034231 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f350fd66-05dc-4069-8975-0da579d09d97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:08:22.036047 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f350fd66-05dc-4069-8975-0da579d09d97 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:08:22.050093 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f350fd66-05dc-4069-8975-0da579d09d97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:08:22.051526 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f350fd66-05dc-4069-8975-0da579d09d97 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:08:22.062750 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1147/2314] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:08:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:08:30.847682 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=6 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:08:30.953218 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdddae5e-6bb9-4026-a78b-dd3cba3a392e service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:08:30.954626 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-fdddae5e-6bb9-4026-a78b-dd3cba3a392e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:08:31.011768 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdddae5e-6bb9-4026-a78b-dd3cba3a392e service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:08:31.015413 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b72569-a8c3-43f2-9df0-5a605c7f2072 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6cc95a563b941ac92ab283aa0d5a5d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:08:31.016107 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b72569-a8c3-43f2-9df0-5a605c7f2072 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:08:31.019694 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b72569-a8c3-43f2-9df0-5a605c7f2072 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:08:31.019694 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b72569-a8c3-43f2-9df0-5a605c7f2072 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:08:31.019694 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-09b72569-a8c3-43f2-9df0-5a605c7f2072 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:08:31.081073 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b72569-a8c3-43f2-9df0-5a605c7f2072 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:08:31.082504 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b72569-a8c3-43f2-9df0-5a605c7f2072 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:08:31.093200 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1168/2315] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:08:30 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:08:32.057783 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=6 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:08:36.749477 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6e9fa4-b211-4a16-84f4-9d5fbf3ef2fc None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:08:36.755169 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6e9fa4-b211-4a16-84f4-9d5fbf3ef2fc None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:08:36.755169 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6e9fa4-b211-4a16-84f4-9d5fbf3ef2fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:08:36.768191 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-3b6e9fa4-b211-4a16-84f4-9d5fbf3ef2fc None None] MFA Rules not processed for user `8e32dbeb85bd4fb889d6dde932c60853`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:08:36.769393 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-3b6e9fa4-b211-4a16-84f4-9d5fbf3ef2fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:08:36.824093 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1148/2316] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:08:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:08:37.715268 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b72569-a8c3-43f2-9df0-5a605c7f2072 service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:08:37.726974 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b72569-a8c3-43f2-9df0-5a605c7f2072 service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:08:37.732031 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1515017-7aa8-4995-a2b3-897936eeb765 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd972ab782b3f4d2aba575314f16dafc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:08:37.732736 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1515017-7aa8-4995-a2b3-897936eeb765 service placement] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:08:37.733025 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1515017-7aa8-4995-a2b3-897936eeb765 service placement] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:08:37.733306 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1515017-7aa8-4995-a2b3-897936eeb765 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:08:37.735129 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a1515017-7aa8-4995-a2b3-897936eeb765 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:08:37.747493 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1515017-7aa8-4995-a2b3-897936eeb765 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:08:37.749246 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1515017-7aa8-4995-a2b3-897936eeb765 service placement] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:08:37.759513 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1169/2317] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:08:37 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:08:40.565183 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6e9fa4-b211-4a16-84f4-9d5fbf3ef2fc None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:08:40.576183 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6e9fa4-b211-4a16-84f4-9d5fbf3ef2fc None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:08:40.579898 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5c98f5-636b-4576-bd09-68862d679f17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:08:40.580580 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5c98f5-636b-4576-bd09-68862d679f17 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:08:40.580929 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5c98f5-636b-4576-bd09-68862d679f17 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:08:40.581120 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5c98f5-636b-4576-bd09-68862d679f17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:08:40.583162 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-8b5c98f5-636b-4576-bd09-68862d679f17 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:08:40.593077 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5c98f5-636b-4576-bd09-68862d679f17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:08:40.594513 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5c98f5-636b-4576-bd09-68862d679f17 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:08:40.602310 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1149/2318] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:08:40 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:08:40.978628 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1515017-7aa8-4995-a2b3-897936eeb765 service placement] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:08:40.980112 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a1515017-7aa8-4995-a2b3-897936eeb765 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:08:40.995202 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1515017-7aa8-4995-a2b3-897936eeb765 service placement] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:08:41.001206 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a64df5c-548a-4966-a3c3-5c130042377c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bd6caa461e84100a3d5fe5af5d80f3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:08:41.001850 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a64df5c-548a-4966-a3c3-5c130042377c service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:08:41.002118 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a64df5c-548a-4966-a3c3-5c130042377c service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:08:41.002395 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a64df5c-548a-4966-a3c3-5c130042377c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:08:41.004177 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8a64df5c-548a-4966-a3c3-5c130042377c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:08:41.014475 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a64df5c-548a-4966-a3c3-5c130042377c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:08:41.016087 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a64df5c-548a-4966-a3c3-5c130042377c service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:08:41.025601 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1170/2319] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:08:40 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:08:41.417722 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5c98f5-636b-4576-bd09-68862d679f17 service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:08:41.454007 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5c98f5-636b-4576-bd09-68862d679f17 service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:08:41.460080 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fb51f1-b862-41b4-b9ee-69dc1fe406de service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bd6caa461e84100a3d5fe5af5d80f3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:08:41.460779 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fb51f1-b862-41b4-b9ee-69dc1fe406de service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:08:41.461056 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fb51f1-b862-41b4-b9ee-69dc1fe406de service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:08:41.461535 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fb51f1-b862-41b4-b9ee-69dc1fe406de service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:08:41.465350 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-94fb51f1-b862-41b4-b9ee-69dc1fe406de service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:08:41.478976 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fb51f1-b862-41b4-b9ee-69dc1fe406de service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:08:41.487227 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fb51f1-b862-41b4-b9ee-69dc1fe406de service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:08:41.493046 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1150/2320] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:08:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:08:51.020968 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=60 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:08:51.487406 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=37 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:08:55.567929 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a64df5c-548a-4966-a3c3-5c130042377c service nova] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:08:55.579716 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a64df5c-548a-4966-a3c3-5c130042377c service nova] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:08:55.585142 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7826b9-76c8-4857-a8bc-c8cdf4ed9d44 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bd6caa461e84100a3d5fe5af5d80f3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:08:55.585770 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7826b9-76c8-4857-a8bc-c8cdf4ed9d44 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:08:55.586060 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7826b9-76c8-4857-a8bc-c8cdf4ed9d44 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:08:55.586337 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7826b9-76c8-4857-a8bc-c8cdf4ed9d44 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:08:55.588101 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-bc7826b9-76c8-4857-a8bc-c8cdf4ed9d44 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:08:55.645340 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7826b9-76c8-4857-a8bc-c8cdf4ed9d44 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:08:55.646841 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7826b9-76c8-4857-a8bc-c8cdf4ed9d44 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:08:55.656014 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1171/2321] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:08:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:09:05.651920 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=28 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:09:06.498825 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22381f78-ec94-4fdc-9c9b-a9f1b32fa486 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:09:06.499352 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22381f78-ec94-4fdc-9c9b-a9f1b32fa486 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:09:06.499455 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22381f78-ec94-4fdc-9c9b-a9f1b32fa486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:09:06.517012 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-22381f78-ec94-4fdc-9c9b-a9f1b32fa486 None None] MFA Rules not processed for user `c71b903e00214894b94b03b3401607a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:09:06.518205 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-22381f78-ec94-4fdc-9c9b-a9f1b32fa486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:09:06.525030 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1151/2322] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:09:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:09:06.823112 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7826b9-76c8-4857-a8bc-c8cdf4ed9d44 service nova] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:09:06.832682 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7826b9-76c8-4857-a8bc-c8cdf4ed9d44 service nova] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:09:06.836245 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552a67c6-2186-4279-b3ab-a726d3e03b55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:09:06.836876 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552a67c6-2186-4279-b3ab-a726d3e03b55 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:09:06.837158 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552a67c6-2186-4279-b3ab-a726d3e03b55 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:09:06.837434 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552a67c6-2186-4279-b3ab-a726d3e03b55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:09:06.839236 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-552a67c6-2186-4279-b3ab-a726d3e03b55 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:09:06.849063 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552a67c6-2186-4279-b3ab-a726d3e03b55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:09:06.850497 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552a67c6-2186-4279-b3ab-a726d3e03b55 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:09:06.860242 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1172/2323] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:09:06 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:09:14.048606 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c708f4c-8201-4dca-8743-be21f52f6363 None None] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:09:14.049147 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c708f4c-8201-4dca-8743-be21f52f6363 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:09:14.049366 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c708f4c-8201-4dca-8743-be21f52f6363 None None] PATH_INFO: `/` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:09:14.050925 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1152/2324] 10.222.0.27 () {54 vars in 921 bytes} [Fri Jun 10 13:09:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 13:09:14.056708 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fbafa9-0244-4b93-949b-cf1658300991 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:09:14.057166 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fbafa9-0244-4b93-949b-cf1658300991 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:09:14.057282 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fbafa9-0244-4b93-949b-cf1658300991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:09:14.075656 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-25fbafa9-0244-4b93-949b-cf1658300991 None None] MFA Rules not processed for user `bc4415461a974c24ad7105a97c7d915b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:09:14.076914 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-25fbafa9-0244-4b93-949b-cf1658300991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:09:14.083736 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1173/2325] 10.222.0.27 () {58 vars in 1018 bytes} [Fri Jun 10 13:09:14 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:09:14.099035 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c708f4c-8201-4dca-8743-be21f52f6363 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:09:14.111000 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c708f4c-8201-4dca-8743-be21f52f6363 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:09:14.118340 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383fe61f-f744-4537-bad5-81f761027e54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:09:14.119012 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383fe61f-f744-4537-bad5-81f761027e54 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:09:14.119310 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383fe61f-f744-4537-bad5-81f761027e54 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:09:14.119602 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383fe61f-f744-4537-bad5-81f761027e54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:09:14.127926 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383fe61f-f744-4537-bad5-81f761027e54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:09:14.129401 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383fe61f-f744-4537-bad5-81f761027e54 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:09:14.137240 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1153/2326] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:09:14 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:09:14.217392 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391388fa-e11a-4d17-87c0-0c6c58cbc6eb None None] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:09:14.217700 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391388fa-e11a-4d17-87c0-0c6c58cbc6eb None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:09:14.217977 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391388fa-e11a-4d17-87c0-0c6c58cbc6eb None None] PATH_INFO: `/` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:09:14.219371 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1174/2327] 10.222.0.27 () {54 vars in 921 bytes} [Fri Jun 10 13:09:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 10 13:09:14.225349 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb28d3fd-5bfc-40d2-ae2f-324c8a43d1fe None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:09:14.225606 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb28d3fd-5bfc-40d2-ae2f-324c8a43d1fe None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:09:14.225893 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb28d3fd-5bfc-40d2-ae2f-324c8a43d1fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:09:14.244460 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-cb28d3fd-5bfc-40d2-ae2f-324c8a43d1fe None None] MFA Rules not processed for user `8bd6caa461e84100a3d5fe5af5d80f3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:09:14.245690 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-cb28d3fd-5bfc-40d2-ae2f-324c8a43d1fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:09:14.255006 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1154/2328] 10.222.0.27 () {58 vars in 1018 bytes} [Fri Jun 10 13:09:14 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:09:14.265125 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391388fa-e11a-4d17-87c0-0c6c58cbc6eb None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:09:14.276320 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391388fa-e11a-4d17-87c0-0c6c58cbc6eb None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:09:14.280522 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22476cba-0fdb-4619-b951-94dc367c4ec3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:09:14.281875 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22476cba-0fdb-4619-b951-94dc367c4ec3 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:09:14.282232 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22476cba-0fdb-4619-b951-94dc367c4ec3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:09:14.282578 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22476cba-0fdb-4619-b951-94dc367c4ec3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:09:14.293476 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22476cba-0fdb-4619-b951-94dc367c4ec3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:09:14.295573 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22476cba-0fdb-4619-b951-94dc367c4ec3 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:09:14.306878 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1175/2329] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:09:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:09:16.111952 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb28d3fd-5bfc-40d2-ae2f-324c8a43d1fe None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:09:16.113538 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-cb28d3fd-5bfc-40d2-ae2f-324c8a43d1fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:09:16.127326 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb28d3fd-5bfc-40d2-ae2f-324c8a43d1fe None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:09:16.132819 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378cae9e-086f-4c5c-9e5f-41f6d427c33e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bd6caa461e84100a3d5fe5af5d80f3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:09:16.133463 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378cae9e-086f-4c5c-9e5f-41f6d427c33e service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:09:16.133743 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378cae9e-086f-4c5c-9e5f-41f6d427c33e service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:09:16.134022 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378cae9e-086f-4c5c-9e5f-41f6d427c33e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:09:16.135759 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-378cae9e-086f-4c5c-9e5f-41f6d427c33e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:09:16.149168 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378cae9e-086f-4c5c-9e5f-41f6d427c33e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:09:16.151281 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378cae9e-086f-4c5c-9e5f-41f6d427c33e service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:09:16.163653 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1155/2330] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:09:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:09:17.680400 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22476cba-0fdb-4619-b951-94dc367c4ec3 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:09:17.689697 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22476cba-0fdb-4619-b951-94dc367c4ec3 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:09:17.693314 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38eb17b-f9a6-4a47-a3cf-ef9109cefb87 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6cc95a563b941ac92ab283aa0d5a5d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:09:17.693944 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38eb17b-f9a6-4a47-a3cf-ef9109cefb87 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:09:17.694233 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38eb17b-f9a6-4a47-a3cf-ef9109cefb87 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:09:17.694513 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38eb17b-f9a6-4a47-a3cf-ef9109cefb87 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:09:17.696319 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-e38eb17b-f9a6-4a47-a3cf-ef9109cefb87 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:09:17.708089 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38eb17b-f9a6-4a47-a3cf-ef9109cefb87 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:09:17.710127 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38eb17b-f9a6-4a47-a3cf-ef9109cefb87 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:09:17.729297 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1176/2331] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:09:17 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:09:22.817686 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378cae9e-086f-4c5c-9e5f-41f6d427c33e service nova] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:09:22.828899 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378cae9e-086f-4c5c-9e5f-41f6d427c33e service nova] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:09:22.834355 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dda9aa-ff87-43bd-9dca-77060b6afb10 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bd6caa461e84100a3d5fe5af5d80f3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:09:22.834998 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dda9aa-ff87-43bd-9dca-77060b6afb10 service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:09:22.835286 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dda9aa-ff87-43bd-9dca-77060b6afb10 service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:09:22.835567 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dda9aa-ff87-43bd-9dca-77060b6afb10 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:09:22.837361 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-67dda9aa-ff87-43bd-9dca-77060b6afb10 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:09:22.850408 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dda9aa-ff87-43bd-9dca-77060b6afb10 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:09:22.852039 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67dda9aa-ff87-43bd-9dca-77060b6afb10 service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:09:22.863930 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1156/2332] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:09:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:09:27.717489 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=21 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:09:32.859441 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=24 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:09:43.376600 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38eb17b-f9a6-4a47-a3cf-ef9109cefb87 service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:09:43.385744 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38eb17b-f9a6-4a47-a3cf-ef9109cefb87 service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:09:43.389360 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956ee288-7e7f-4e07-9360-1110f502f350 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:09:43.390012 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956ee288-7e7f-4e07-9360-1110f502f350 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:09:43.390296 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956ee288-7e7f-4e07-9360-1110f502f350 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:09:43.390575 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956ee288-7e7f-4e07-9360-1110f502f350 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:09:43.392413 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-956ee288-7e7f-4e07-9360-1110f502f350 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:09:43.454614 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956ee288-7e7f-4e07-9360-1110f502f350 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:09:43.456054 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956ee288-7e7f-4e07-9360-1110f502f350 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:09:43.465445 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1177/2333] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:09:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:09:47.035162 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dda9aa-ff87-43bd-9dca-77060b6afb10 service nova] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:09:47.047389 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67dda9aa-ff87-43bd-9dca-77060b6afb10 service nova] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:09:47.052554 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8919d5-8856-4032-acf2-0474de1a40e4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd972ab782b3f4d2aba575314f16dafc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:09:47.053851 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8919d5-8856-4032-acf2-0474de1a40e4 service placement] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:09:47.054399 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8919d5-8856-4032-acf2-0474de1a40e4 service placement] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:09:47.054897 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8919d5-8856-4032-acf2-0474de1a40e4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:09:47.056987 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-9c8919d5-8856-4032-acf2-0474de1a40e4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:09:47.069871 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8919d5-8856-4032-acf2-0474de1a40e4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:09:47.071981 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8919d5-8856-4032-acf2-0474de1a40e4 service placement] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:09:47.083490 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1157/2334] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:09:47 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:09:47.161175 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956ee288-7e7f-4e07-9360-1110f502f350 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:09:47.162631 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-956ee288-7e7f-4e07-9360-1110f502f350 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:09:47.175592 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956ee288-7e7f-4e07-9360-1110f502f350 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:09:47.180337 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d307076b-9eb5-4402-9c36-d118f69c2cde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:09:47.181218 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d307076b-9eb5-4402-9c36-d118f69c2cde admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:09:47.181519 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d307076b-9eb5-4402-9c36-d118f69c2cde admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:09:47.181797 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d307076b-9eb5-4402-9c36-d118f69c2cde admin admin] PATH_INFO: `/v3/users/e0b63da5bf0b4410acc9ff573064f8ec` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:09:47.182982 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d307076b-9eb5-4402-9c36-d118f69c2cde admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0b63da5bf0b4410acc9ff573064f8ec)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:09:47.184927 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d307076b-9eb5-4402-9c36-d118f69c2cde admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:09:47.350289 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d307076b-9eb5-4402-9c36-d118f69c2cde admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0b63da5bf0b4410acc9ff573064f8ec'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:09:47.357316 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d307076b-9eb5-4402-9c36-d118f69c2cde admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0b63da5bf0b4410acc9ff573064f8ec'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:09:47.405337 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d307076b-9eb5-4402-9c36-d118f69c2cde admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0b63da5bf0b4410acc9ff573064f8ec'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:09:47.405898 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d307076b-9eb5-4402-9c36-d118f69c2cde admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0b63da5bf0b4410acc9ff573064f8ec'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:09:47.419289 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d307076b-9eb5-4402-9c36-d118f69c2cde admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0b63da5bf0b4410acc9ff573064f8ec'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:09:47.425946 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1178/2335] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:09:47 2022] DELETE /identity/v3/users/e0b63da5bf0b4410acc9ff573064f8ec => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:09:47.659134 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8919d5-8856-4032-acf2-0474de1a40e4 service placement] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:09:47.669697 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8919d5-8856-4032-acf2-0474de1a40e4 service placement] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:09:47.725751 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b71bbd-87a2-4cfb-be30-3f45a4942706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:09:47.726612 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b71bbd-87a2-4cfb-be30-3f45a4942706 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:09:47.726893 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b71bbd-87a2-4cfb-be30-3f45a4942706 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:09:47.727168 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b71bbd-87a2-4cfb-be30-3f45a4942706 admin admin] PATH_INFO: `/v3/projects/bad59dfe0332406d973508224a0162e4` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:09:47.728364 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b71bbd-87a2-4cfb-be30-3f45a4942706 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bad59dfe0332406d973508224a0162e4)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:09:47.730298 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b71bbd-87a2-4cfb-be30-3f45a4942706 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:09:47.839288 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c6b71bbd-87a2-4cfb-be30-3f45a4942706 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bad59dfe0332406d973508224a0162e4', 'request_id': 'req-c6b71bbd-87a2-4cfb-be30-3f45a4942706'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:09:47.844706 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c6b71bbd-87a2-4cfb-be30-3f45a4942706 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bad59dfe0332406d973508224a0162e4', 'request_id': 'req-c6b71bbd-87a2-4cfb-be30-3f45a4942706'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:09:47.850681 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c6b71bbd-87a2-4cfb-be30-3f45a4942706 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bad59dfe0332406d973508224a0162e4', 'request_id': 'req-c6b71bbd-87a2-4cfb-be30-3f45a4942706'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:09:47.908771 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c6b71bbd-87a2-4cfb-be30-3f45a4942706 admin admin] The token cache is being invalidate because project bad59dfe0332406d973508224a0162e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:09:47.992895 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c6b71bbd-87a2-4cfb-be30-3f45a4942706 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:09:47.992895 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-c6b71bbd-87a2-4cfb-be30-3f45a4942706 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:09:47.992895 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1158/2336] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:09:47 2022] DELETE /identity/v3/projects/bad59dfe0332406d973508224a0162e4 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:09:47.993279 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af7e7b8-d46d-4d96-bf04-187ee586deac None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:09:47.993279 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af7e7b8-d46d-4d96-bf04-187ee586deac None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:09:47.993279 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af7e7b8-d46d-4d96-bf04-187ee586deac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:09:47.993279 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-7af7e7b8-d46d-4d96-bf04-187ee586deac None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:09:47.993279 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-7af7e7b8-d46d-4d96-bf04-187ee586deac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:09:48.003731 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1179/2337] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:09:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:09:48.011556 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602c6343-9f1b-4c0a-948a-4fb14c03bedc None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:09:48.011851 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602c6343-9f1b-4c0a-948a-4fb14c03bedc None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:09:48.012167 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602c6343-9f1b-4c0a-948a-4fb14c03bedc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:09:48.032147 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-602c6343-9f1b-4c0a-948a-4fb14c03bedc None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:09:48.033344 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-602c6343-9f1b-4c0a-948a-4fb14c03bedc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:09:48.041151 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1159/2338] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:09:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:09:48.045146 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af7e7b8-d46d-4d96-bf04-187ee586deac None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:09:48.054854 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af7e7b8-d46d-4d96-bf04-187ee586deac None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:09:48.059430 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf770e8-5611-41c2-8e73-547ff282c22b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:09:48.060104 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf770e8-5611-41c2-8e73-547ff282c22b admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:09:48.060388 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf770e8-5611-41c2-8e73-547ff282c22b admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:09:48.060659 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf770e8-5611-41c2-8e73-547ff282c22b admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:09:48.061383 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf770e8-5611-41c2-8e73-547ff282c22b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 13:09:48.061687 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf770e8-5611-41c2-8e73-547ff282c22b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:09:48.063854 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf770e8-5611-41c2-8e73-547ff282c22b admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:09:48.065323 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:09:48.075469 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1180/2339] 10.222.0.27 () {58 vars in 1152 bytes} [Fri Jun 10 13:09:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 13:09:48.080340 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602c6343-9f1b-4c0a-948a-4fb14c03bedc None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:09:48.089579 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602c6343-9f1b-4c0a-948a-4fb14c03bedc None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:09:48.094284 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5232052-0685-4f4d-a080-f3833984ccf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:09:48.094923 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5232052-0685-4f4d-a080-f3833984ccf2 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:09:48.095226 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5232052-0685-4f4d-a080-f3833984ccf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:09:48.095501 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5232052-0685-4f4d-a080-f3833984ccf2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:09:48.096229 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5232052-0685-4f4d-a080-f3833984ccf2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:09:48.098176 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5232052-0685-4f4d-a080-f3833984ccf2 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:09:48.195064 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1160/2340] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:09:48 2022] POST /identity/v3/projects => generated 383 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 10 13:09:48.198948 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf770e8-5611-41c2-8e73-547ff282c22b admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:09:48.209036 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf770e8-5611-41c2-8e73-547ff282c22b admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:09:48.263777 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b72952f-c196-4aa3-bb94-89439c2b93a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:09:48.264394 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b72952f-c196-4aa3-bb94-89439c2b93a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:09:48.264696 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b72952f-c196-4aa3-bb94-89439c2b93a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:09:48.264956 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b72952f-c196-4aa3-bb94-89439c2b93a7 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:09:48.265642 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b72952f-c196-4aa3-bb94-89439c2b93a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:09:48.267527 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b72952f-c196-4aa3-bb94-89439c2b93a7 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:09:48.343053 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1181/2341] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:09:48 2022] POST /identity/v3/users => generated 292 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 10 13:09:48.346829 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5232052-0685-4f4d-a080-f3833984ccf2 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:09:48.360276 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5232052-0685-4f4d-a080-f3833984ccf2 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:09:48.365420 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e635839-3e9f-41d7-a419-714978efc646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:09:48.366090 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e635839-3e9f-41d7-a419-714978efc646 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:09:48.366396 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e635839-3e9f-41d7-a419-714978efc646 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:09:48.366668 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e635839-3e9f-41d7-a419-714978efc646 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:09:48.367395 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e635839-3e9f-41d7-a419-714978efc646 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:09:48.369062 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e635839-3e9f-41d7-a419-714978efc646 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:09:48.389530 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1161/2342] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:09:48 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:09:48.390760 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b72952f-c196-4aa3-bb94-89439c2b93a7 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:09:48.401178 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b72952f-c196-4aa3-bb94-89439c2b93a7 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:09:48.406094 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b041c9-cd97-4709-bd70-d4dffee7f34d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:09:48.406757 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b041c9-cd97-4709-bd70-d4dffee7f34d admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:09:48.407035 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b041c9-cd97-4709-bd70-d4dffee7f34d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:09:48.407313 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b041c9-cd97-4709-bd70-d4dffee7f34d admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:09:48.408144 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b041c9-cd97-4709-bd70-d4dffee7f34d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:09:48.409679 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b041c9-cd97-4709-bd70-d4dffee7f34d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:09:48.420483 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1182/2343] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:09:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:09:48.425288 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e635839-3e9f-41d7-a419-714978efc646 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:09:48.435361 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e635839-3e9f-41d7-a419-714978efc646 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:09:48.440764 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ed7b56-4c11-4b9a-8673-3e5d7d587328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:09:48.441635 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ed7b56-4c11-4b9a-8673-3e5d7d587328 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:09:48.441918 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ed7b56-4c11-4b9a-8673-3e5d7d587328 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:09:48.442193 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ed7b56-4c11-4b9a-8673-3e5d7d587328 admin admin] PATH_INFO: `/v3/projects/afa4b780bfec44a6ac5ceb0d259dd55a/users/c1db8f1ebe7b487dbd5217df19c3110b/roles/43106c664e1b40de984c0c62701149c8` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:09:48.456581 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ed7b56-4c11-4b9a-8673-3e5d7d587328 admin admin] RBAC: Authorizing `identity:create_grant(project_id=afa4b780bfec44a6ac5ceb0d259dd55a, user_id=c1db8f1ebe7b487dbd5217df19c3110b, role_id=43106c664e1b40de984c0c62701149c8)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:09:48.459202 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ed7b56-4c11-4b9a-8673-3e5d7d587328 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:09:48.524959 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1162/2344] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:09:48 2022] PUT /identity/v3/projects/afa4b780bfec44a6ac5ceb0d259dd55a/users/c1db8f1ebe7b487dbd5217df19c3110b/roles/43106c664e1b40de984c0c62701149c8 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 10 13:09:48.535178 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbae913-9485-4d1f-995b-83d64dcba5ed None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:09:48.535472 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbae913-9485-4d1f-995b-83d64dcba5ed None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:09:48.535751 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbae913-9485-4d1f-995b-83d64dcba5ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:09:48.561840 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-4dbae913-9485-4d1f-995b-83d64dcba5ed None None] MFA Rules not processed for user `c1db8f1ebe7b487dbd5217df19c3110b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:09:48.563682 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-4dbae913-9485-4d1f-995b-83d64dcba5ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:09:48.626367 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1183/2345] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:09:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:09:48.637389 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ed7b56-4c11-4b9a-8673-3e5d7d587328 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:09:48.640359 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-95ed7b56-4c11-4b9a-8673-3e5d7d587328 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:09:48.646431 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:09:48.713193 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ed7b56-4c11-4b9a-8673-3e5d7d587328 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:09:48.719025 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8908412-2e0d-411b-85c9-f853e007d9e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bd6caa461e84100a3d5fe5af5d80f3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:09:48.719674 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8908412-2e0d-411b-85c9-f853e007d9e6 service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:09:48.719989 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8908412-2e0d-411b-85c9-f853e007d9e6 service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:09:48.720269 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8908412-2e0d-411b-85c9-f853e007d9e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:09:48.728218 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8908412-2e0d-411b-85c9-f853e007d9e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:09:48.729778 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8908412-2e0d-411b-85c9-f853e007d9e6 service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:09:48.738471 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1163/2346] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:09:48 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:09:54.565952 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbae913-9485-4d1f-995b-83d64dcba5ed None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:09:54.627059 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbae913-9485-4d1f-995b-83d64dcba5ed None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:09:54.631569 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427c07ae-6bf8-490e-8b72-1d13399e3b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:09:54.632461 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427c07ae-6bf8-490e-8b72-1d13399e3b20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:09:54.632749 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427c07ae-6bf8-490e-8b72-1d13399e3b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:09:54.633027 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427c07ae-6bf8-490e-8b72-1d13399e3b20 admin admin] PATH_INFO: `/v3/users/c1db8f1ebe7b487dbd5217df19c3110b` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:09:54.634203 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427c07ae-6bf8-490e-8b72-1d13399e3b20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1db8f1ebe7b487dbd5217df19c3110b)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:09:54.636145 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427c07ae-6bf8-490e-8b72-1d13399e3b20 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:09:54.638362 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:09:54.812722 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-427c07ae-6bf8-490e-8b72-1d13399e3b20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1db8f1ebe7b487dbd5217df19c3110b'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:09:54.819399 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-427c07ae-6bf8-490e-8b72-1d13399e3b20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1db8f1ebe7b487dbd5217df19c3110b'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:09:54.873967 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-427c07ae-6bf8-490e-8b72-1d13399e3b20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1db8f1ebe7b487dbd5217df19c3110b'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:09:54.874583 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-427c07ae-6bf8-490e-8b72-1d13399e3b20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1db8f1ebe7b487dbd5217df19c3110b'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:09:54.887657 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-427c07ae-6bf8-490e-8b72-1d13399e3b20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1db8f1ebe7b487dbd5217df19c3110b'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:09:54.893946 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1184/2347] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:09:54 2022] DELETE /identity/v3/users/c1db8f1ebe7b487dbd5217df19c3110b => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:09:54.898405 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934bb9ba-8b6f-4e9b-817e-cf05617c7936 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:09:54.898853 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934bb9ba-8b6f-4e9b-817e-cf05617c7936 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:09:54.898965 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934bb9ba-8b6f-4e9b-817e-cf05617c7936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:09:54.916533 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-934bb9ba-8b6f-4e9b-817e-cf05617c7936 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:09:54.917730 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-934bb9ba-8b6f-4e9b-817e-cf05617c7936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:09:54.974488 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1164/2348] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:09:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 10 13:09:54.983881 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427c07ae-6bf8-490e-8b72-1d13399e3b20 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:09:54.985284 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-427c07ae-6bf8-490e-8b72-1d13399e3b20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:09:55.042479 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427c07ae-6bf8-490e-8b72-1d13399e3b20 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:09:55.046095 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ce1561-f453-4026-bea9-9f8d27d4255d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:09:55.046728 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ce1561-f453-4026-bea9-9f8d27d4255d service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:09:55.047011 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ce1561-f453-4026-bea9-9f8d27d4255d service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:09:55.047287 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ce1561-f453-4026-bea9-9f8d27d4255d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:09:55.056135 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ce1561-f453-4026-bea9-9f8d27d4255d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:09:55.057547 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ce1561-f453-4026-bea9-9f8d27d4255d service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:09:55.066833 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1185/2349] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:09:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 10 13:09:55.504427 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934bb9ba-8b6f-4e9b-817e-cf05617c7936 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:09:55.505875 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-934bb9ba-8b6f-4e9b-817e-cf05617c7936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:09:55.520433 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934bb9ba-8b6f-4e9b-817e-cf05617c7936 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:09:55.525545 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeee9f0-3a62-45c8-9a0b-2126f0fcc94b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:09:55.526439 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeee9f0-3a62-45c8-9a0b-2126f0fcc94b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:09:55.526984 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeee9f0-3a62-45c8-9a0b-2126f0fcc94b admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:09:55.527260 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeee9f0-3a62-45c8-9a0b-2126f0fcc94b admin admin] PATH_INFO: `/v3/projects/afa4b780bfec44a6ac5ceb0d259dd55a` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:09:55.528892 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daeee9f0-3a62-45c8-9a0b-2126f0fcc94b admin admin] RBAC: Authorizing `identity:delete_project(project_id=afa4b780bfec44a6ac5ceb0d259dd55a)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:09:55.530817 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daeee9f0-3a62-45c8-9a0b-2126f0fcc94b admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:09:55.647077 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-daeee9f0-3a62-45c8-9a0b-2126f0fcc94b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afa4b780bfec44a6ac5ceb0d259dd55a', 'request_id': 'req-daeee9f0-3a62-45c8-9a0b-2126f0fcc94b'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:09:55.653205 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-daeee9f0-3a62-45c8-9a0b-2126f0fcc94b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afa4b780bfec44a6ac5ceb0d259dd55a', 'request_id': 'req-daeee9f0-3a62-45c8-9a0b-2126f0fcc94b'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:09:55.660400 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-daeee9f0-3a62-45c8-9a0b-2126f0fcc94b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afa4b780bfec44a6ac5ceb0d259dd55a', 'request_id': 'req-daeee9f0-3a62-45c8-9a0b-2126f0fcc94b'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:09:55.768433 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-daeee9f0-3a62-45c8-9a0b-2126f0fcc94b admin admin] The token cache is being invalidate because project afa4b780bfec44a6ac5ceb0d259dd55a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:09:55.768433 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-daeee9f0-3a62-45c8-9a0b-2126f0fcc94b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:09:55.768433 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-daeee9f0-3a62-45c8-9a0b-2126f0fcc94b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:09:55.768433 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1165/2350] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:09:55 2022] DELETE /identity/v3/projects/afa4b780bfec44a6ac5ceb0d259dd55a => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:09:55.772441 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f8f49e-50a8-48ab-b516-21b8e4f3e384 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:09:55.772923 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f8f49e-50a8-48ab-b516-21b8e4f3e384 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:09:55.773017 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f8f49e-50a8-48ab-b516-21b8e4f3e384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:09:55.790418 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-04f8f49e-50a8-48ab-b516-21b8e4f3e384 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:09:55.791598 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-04f8f49e-50a8-48ab-b516-21b8e4f3e384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:09:55.849683 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1186/2351] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:09:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:09:55.858407 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5520b1d1-5b84-449f-922e-a3c305b00c94 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:09:55.858699 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5520b1d1-5b84-449f-922e-a3c305b00c94 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:09:55.858984 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5520b1d1-5b84-449f-922e-a3c305b00c94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:09:55.878855 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-5520b1d1-5b84-449f-922e-a3c305b00c94 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:09:55.880129 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5520b1d1-5b84-449f-922e-a3c305b00c94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:09:55.888115 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1166/2352] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:09:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:09:55.892250 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f8f49e-50a8-48ab-b516-21b8e4f3e384 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:09:55.902209 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f8f49e-50a8-48ab-b516-21b8e4f3e384 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:09:55.906967 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c312d642-e662-429d-9cc3-14f585926775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:09:55.907623 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c312d642-e662-429d-9cc3-14f585926775 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:09:55.907968 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c312d642-e662-429d-9cc3-14f585926775 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:09:55.908249 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c312d642-e662-429d-9cc3-14f585926775 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:09:55.908980 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c312d642-e662-429d-9cc3-14f585926775 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 13:09:55.909291 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c312d642-e662-429d-9cc3-14f585926775 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:09:55.910878 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c312d642-e662-429d-9cc3-14f585926775 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:09:55.922596 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1187/2353] 10.222.0.27 () {58 vars in 1152 bytes} [Fri Jun 10 13:09:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 13:09:55.927154 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5520b1d1-5b84-449f-922e-a3c305b00c94 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:09:55.936365 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5520b1d1-5b84-449f-922e-a3c305b00c94 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:09:55.941633 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4a0d55-adcb-4797-8a3c-62204b7a3c7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:09:55.942275 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4a0d55-adcb-4797-8a3c-62204b7a3c7c admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:09:55.942559 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4a0d55-adcb-4797-8a3c-62204b7a3c7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:09:55.942835 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4a0d55-adcb-4797-8a3c-62204b7a3c7c admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:09:55.943517 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4a0d55-adcb-4797-8a3c-62204b7a3c7c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:09:55.945429 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4a0d55-adcb-4797-8a3c-62204b7a3c7c admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:09:55.948608 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:09:56.040929 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1167/2354] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:09:55 2022] POST /identity/v3/projects => generated 405 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:09:56.044869 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c312d642-e662-429d-9cc3-14f585926775 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:09:56.055427 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c312d642-e662-429d-9cc3-14f585926775 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:09:56.063090 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:09:56.110052 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf14e4a-d0b7-432e-b7d0-cbba7ab90df2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:09:56.110717 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf14e4a-d0b7-432e-b7d0-cbba7ab90df2 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:09:56.111005 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf14e4a-d0b7-432e-b7d0-cbba7ab90df2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:09:56.111283 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf14e4a-d0b7-432e-b7d0-cbba7ab90df2 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:09:56.112678 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf14e4a-d0b7-432e-b7d0-cbba7ab90df2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:09:56.118124 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf14e4a-d0b7-432e-b7d0-cbba7ab90df2 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:09:56.222573 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1188/2355] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:09:56 2022] POST /identity/v3/users => generated 303 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:09:56.226884 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4a0d55-adcb-4797-8a3c-62204b7a3c7c admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:09:56.238057 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4a0d55-adcb-4797-8a3c-62204b7a3c7c admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:09:56.242680 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43867f37-2a1a-45db-b216-1b653336321e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:09:56.243382 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43867f37-2a1a-45db-b216-1b653336321e admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:09:56.243669 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43867f37-2a1a-45db-b216-1b653336321e admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:09:56.243997 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43867f37-2a1a-45db-b216-1b653336321e admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:09:56.244729 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43867f37-2a1a-45db-b216-1b653336321e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:09:56.246363 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43867f37-2a1a-45db-b216-1b653336321e admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:09:56.256360 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1168/2356] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:09:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:09:56.260193 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf14e4a-d0b7-432e-b7d0-cbba7ab90df2 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:09:56.269952 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf14e4a-d0b7-432e-b7d0-cbba7ab90df2 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:09:56.274551 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69db00a0-6b69-4dc7-a986-104422c4ab9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:09:56.275196 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69db00a0-6b69-4dc7-a986-104422c4ab9d admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:09:56.275483 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69db00a0-6b69-4dc7-a986-104422c4ab9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:09:56.275771 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69db00a0-6b69-4dc7-a986-104422c4ab9d admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:09:56.276526 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69db00a0-6b69-4dc7-a986-104422c4ab9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:09:56.278385 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69db00a0-6b69-4dc7-a986-104422c4ab9d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:09:56.288586 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1189/2357] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:09:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:09:56.292418 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43867f37-2a1a-45db-b216-1b653336321e admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:09:56.302731 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43867f37-2a1a-45db-b216-1b653336321e admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:09:56.307288 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b27df42-3357-439f-80b4-d5a3a2977e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:09:56.308190 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b27df42-3357-439f-80b4-d5a3a2977e18 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:09:56.308476 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b27df42-3357-439f-80b4-d5a3a2977e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:09:56.308755 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b27df42-3357-439f-80b4-d5a3a2977e18 admin admin] PATH_INFO: `/v3/projects/d002c6244aa448ebb36438ad64585995/users/a471149d222046f6bfec235d789e5bdb/roles/43106c664e1b40de984c0c62701149c8` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:09:56.322880 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b27df42-3357-439f-80b4-d5a3a2977e18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d002c6244aa448ebb36438ad64585995, user_id=a471149d222046f6bfec235d789e5bdb, role_id=43106c664e1b40de984c0c62701149c8)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:09:56.325532 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b27df42-3357-439f-80b4-d5a3a2977e18 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:09:56.394187 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1169/2358] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:09:56 2022] PUT /identity/v3/projects/d002c6244aa448ebb36438ad64585995/users/a471149d222046f6bfec235d789e5bdb/roles/43106c664e1b40de984c0c62701149c8 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:09:56.404536 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d2c33d-ffbf-46b8-b0ec-a9ebda5a692e None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:09:56.404831 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d2c33d-ffbf-46b8-b0ec-a9ebda5a692e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:09:56.405113 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d2c33d-ffbf-46b8-b0ec-a9ebda5a692e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:09:56.433214 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-a4d2c33d-ffbf-46b8-b0ec-a9ebda5a692e None None] MFA Rules not processed for user `a471149d222046f6bfec235d789e5bdb`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:09:56.434513 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a4d2c33d-ffbf-46b8-b0ec-a9ebda5a692e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:09:56.497556 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1190/2359] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:09:56 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:09:56.502002 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b27df42-3357-439f-80b4-d5a3a2977e18 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:09:56.565132 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b27df42-3357-439f-80b4-d5a3a2977e18 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:09:56.569595 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d143ec4d-d2d4-49d1-a70f-8a23367a2416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:09:56.570245 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d143ec4d-d2d4-49d1-a70f-8a23367a2416 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:09:56.570525 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d143ec4d-d2d4-49d1-a70f-8a23367a2416 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:09:56.570798 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d143ec4d-d2d4-49d1-a70f-8a23367a2416 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:09:56.571498 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d143ec4d-d2d4-49d1-a70f-8a23367a2416 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:09:56.573418 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d143ec4d-d2d4-49d1-a70f-8a23367a2416 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:09:56.651583 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1170/2360] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:09:56 2022] POST /identity/v3/projects => generated 407 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:09:56.655826 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d2c33d-ffbf-46b8-b0ec-a9ebda5a692e None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:09:56.666926 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d2c33d-ffbf-46b8-b0ec-a9ebda5a692e None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:09:56.722759 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d493aa5e-99e7-4876-a54b-c9b5449821ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:09:56.723525 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d493aa5e-99e7-4876-a54b-c9b5449821ad admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:09:56.723824 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d493aa5e-99e7-4876-a54b-c9b5449821ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:09:56.724127 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d493aa5e-99e7-4876-a54b-c9b5449821ad admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:09:56.724810 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d493aa5e-99e7-4876-a54b-c9b5449821ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:09:56.726707 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d493aa5e-99e7-4876-a54b-c9b5449821ad admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:09:56.800647 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1191/2361] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:09:56 2022] POST /identity/v3/users => generated 310 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:09:56.804591 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d143ec4d-d2d4-49d1-a70f-8a23367a2416 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:09:56.814840 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d143ec4d-d2d4-49d1-a70f-8a23367a2416 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:09:56.819660 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7825ea54-21de-4bb4-8b3f-6fa87c796691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:09:56.820290 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7825ea54-21de-4bb4-8b3f-6fa87c796691 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:09:56.820555 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7825ea54-21de-4bb4-8b3f-6fa87c796691 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:09:56.820966 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7825ea54-21de-4bb4-8b3f-6fa87c796691 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:09:56.821558 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7825ea54-21de-4bb4-8b3f-6fa87c796691 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:09:56.823205 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7825ea54-21de-4bb4-8b3f-6fa87c796691 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:09:56.838349 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1171/2362] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:09:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:09:56.842475 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d493aa5e-99e7-4876-a54b-c9b5449821ad admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:09:56.853165 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d493aa5e-99e7-4876-a54b-c9b5449821ad admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:09:56.860266 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea6ef33-71ff-40b9-bb34-b2106e508fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:09:56.861228 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea6ef33-71ff-40b9-bb34-b2106e508fd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:09:56.861507 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea6ef33-71ff-40b9-bb34-b2106e508fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:09:56.863148 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea6ef33-71ff-40b9-bb34-b2106e508fd1 admin admin] PATH_INFO: `/v3/projects/efe70c22fd4b40c2a2c5a10e31ece4c4/users/47cd8fd04e9e4d01baf3e455ff836327/roles/b43d5a8247e244869c2c70f986f42e8a` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:09:56.869796 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:09:56.883202 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea6ef33-71ff-40b9-bb34-b2106e508fd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=efe70c22fd4b40c2a2c5a10e31ece4c4, user_id=47cd8fd04e9e4d01baf3e455ff836327, role_id=b43d5a8247e244869c2c70f986f42e8a)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:09:56.886074 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea6ef33-71ff-40b9-bb34-b2106e508fd1 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:09:56.971903 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1192/2363] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:09:56 2022] PUT /identity/v3/projects/efe70c22fd4b40c2a2c5a10e31ece4c4/users/47cd8fd04e9e4d01baf3e455ff836327/roles/b43d5a8247e244869c2c70f986f42e8a => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 10 13:09:56.975773 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7825ea54-21de-4bb4-8b3f-6fa87c796691 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:09:56.985659 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7825ea54-21de-4bb4-8b3f-6fa87c796691 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:09:57.040941 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5daac63-4fba-4483-9907-504307cb8aba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:09:57.041637 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5daac63-4fba-4483-9907-504307cb8aba admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:09:57.041925 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5daac63-4fba-4483-9907-504307cb8aba admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:09:57.042207 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5daac63-4fba-4483-9907-504307cb8aba admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:09:57.042945 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5daac63-4fba-4483-9907-504307cb8aba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:09:57.044577 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5daac63-4fba-4483-9907-504307cb8aba admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:09:57.054989 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1172/2364] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:09:56 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:09:57.059005 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea6ef33-71ff-40b9-bb34-b2106e508fd1 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:09:57.069151 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea6ef33-71ff-40b9-bb34-b2106e508fd1 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:09:57.073878 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba933b3-40e8-4561-9665-cbe4a63edefe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:09:57.074691 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba933b3-40e8-4561-9665-cbe4a63edefe admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:09:57.074977 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba933b3-40e8-4561-9665-cbe4a63edefe admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:09:57.075257 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba933b3-40e8-4561-9665-cbe4a63edefe admin admin] PATH_INFO: `/v3/projects/efe70c22fd4b40c2a2c5a10e31ece4c4/users/47cd8fd04e9e4d01baf3e455ff836327/roles/b43d5a8247e244869c2c70f986f42e8a` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:09:57.077243 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba933b3-40e8-4561-9665-cbe4a63edefe admin admin] RBAC: Authorizing `identity:create_grant(project_id=efe70c22fd4b40c2a2c5a10e31ece4c4, user_id=47cd8fd04e9e4d01baf3e455ff836327, role_id=b43d5a8247e244869c2c70f986f42e8a)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:09:57.079834 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba933b3-40e8-4561-9665-cbe4a63edefe admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:09:57.089744 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1193/2365] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:09:57 2022] PUT /identity/v3/projects/efe70c22fd4b40c2a2c5a10e31ece4c4/users/47cd8fd04e9e4d01baf3e455ff836327/roles/b43d5a8247e244869c2c70f986f42e8a => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:09:57.100233 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3c108c-fd80-435f-a2f4-78c0ffcc17a9 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:09:57.100603 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3c108c-fd80-435f-a2f4-78c0ffcc17a9 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:09:57.100923 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3c108c-fd80-435f-a2f4-78c0ffcc17a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:09:57.128454 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-3b3c108c-fd80-435f-a2f4-78c0ffcc17a9 None None] MFA Rules not processed for user `47cd8fd04e9e4d01baf3e455ff836327`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:09:57.133199 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-3b3c108c-fd80-435f-a2f4-78c0ffcc17a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:09:57.137758 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:09:57.195789 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1173/2366] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:09:57 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:09:57.206843 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba933b3-40e8-4561-9665-cbe4a63edefe admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:09:57.208520 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-dba933b3-40e8-4561-9665-cbe4a63edefe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:09:57.264152 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba933b3-40e8-4561-9665-cbe4a63edefe admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:09:57.267731 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66a3704-7475-4042-bf64-fef780813509 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6cc95a563b941ac92ab283aa0d5a5d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:09:57.268904 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66a3704-7475-4042-bf64-fef780813509 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:09:57.269185 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66a3704-7475-4042-bf64-fef780813509 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:09:57.269457 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66a3704-7475-4042-bf64-fef780813509 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:09:57.278973 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66a3704-7475-4042-bf64-fef780813509 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:09:57.280478 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66a3704-7475-4042-bf64-fef780813509 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:09:57.290506 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1194/2367] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:09:57 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:09:57.620714 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3c108c-fd80-435f-a2f4-78c0ffcc17a9 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:09:57.633528 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3c108c-fd80-435f-a2f4-78c0ffcc17a9 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:09:57.637117 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0f6442-9beb-4fdb-baa4-7c5cd26daa1a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6cc95a563b941ac92ab283aa0d5a5d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:09:57.637762 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0f6442-9beb-4fdb-baa4-7c5cd26daa1a service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:09:57.638279 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0f6442-9beb-4fdb-baa4-7c5cd26daa1a service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:09:57.638556 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0f6442-9beb-4fdb-baa4-7c5cd26daa1a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:09:57.706474 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0f6442-9beb-4fdb-baa4-7c5cd26daa1a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:09:57.707854 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0f6442-9beb-4fdb-baa4-7c5cd26daa1a service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:09:57.719009 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1174/2368] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:09:57 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:10:03.252894 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4384ccd-01f9-4c3e-9260-7a854ab46d87 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:03.253410 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4384ccd-01f9-4c3e-9260-7a854ab46d87 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:03.253513 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4384ccd-01f9-4c3e-9260-7a854ab46d87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:03.271621 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-d4384ccd-01f9-4c3e-9260-7a854ab46d87 None None] MFA Rules not processed for user `634e6116f688494c9e11d59479f94a43`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:10:03.272854 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d4384ccd-01f9-4c3e-9260-7a854ab46d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:10:03.326495 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1195/2369] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:10:03 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:10:03.622073 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0f6442-9beb-4fdb-baa4-7c5cd26daa1a service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:10:03.623459 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-2d0f6442-9beb-4fdb-baa4-7c5cd26daa1a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:10:03.680647 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0f6442-9beb-4fdb-baa4-7c5cd26daa1a service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:10:03.684370 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ebb631-6a23-4aea-a8ca-3f93d99ba279 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:10:03.685040 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ebb631-6a23-4aea-a8ca-3f93d99ba279 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:03.685379 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ebb631-6a23-4aea-a8ca-3f93d99ba279 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:03.685657 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ebb631-6a23-4aea-a8ca-3f93d99ba279 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:03.687457 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-79ebb631-6a23-4aea-a8ca-3f93d99ba279 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:10:03.697325 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ebb631-6a23-4aea-a8ca-3f93d99ba279 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:10:03.698736 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ebb631-6a23-4aea-a8ca-3f93d99ba279 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:10:03.707413 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1175/2370] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:10:03 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:10:04.712954 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4384ccd-01f9-4c3e-9260-7a854ab46d87 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:10:04.775264 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4384ccd-01f9-4c3e-9260-7a854ab46d87 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:10:04.780079 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cfb570-a8e6-4bfb-9137-8b39aad8a950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:10:04.780991 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cfb570-a8e6-4bfb-9137-8b39aad8a950 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:04.781288 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cfb570-a8e6-4bfb-9137-8b39aad8a950 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:04.781566 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cfb570-a8e6-4bfb-9137-8b39aad8a950 admin admin] PATH_INFO: `/v3/users/a471149d222046f6bfec235d789e5bdb` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:04.782762 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cfb570-a8e6-4bfb-9137-8b39aad8a950 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a471149d222046f6bfec235d789e5bdb)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:10:04.785315 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cfb570-a8e6-4bfb-9137-8b39aad8a950 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:10:04.808149 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:10:05.024599 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-50cfb570-a8e6-4bfb-9137-8b39aad8a950 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a471149d222046f6bfec235d789e5bdb'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:05.031549 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-50cfb570-a8e6-4bfb-9137-8b39aad8a950 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a471149d222046f6bfec235d789e5bdb'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:05.097429 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-50cfb570-a8e6-4bfb-9137-8b39aad8a950 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a471149d222046f6bfec235d789e5bdb'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:05.098056 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-50cfb570-a8e6-4bfb-9137-8b39aad8a950 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a471149d222046f6bfec235d789e5bdb'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:05.110481 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-50cfb570-a8e6-4bfb-9137-8b39aad8a950 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a471149d222046f6bfec235d789e5bdb'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:05.117124 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1196/2371] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:10:04 2022] DELETE /identity/v3/users/a471149d222046f6bfec235d789e5bdb => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:10:05.121811 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd002f2d-70ea-4eac-8573-24c959dba998 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:05.122333 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd002f2d-70ea-4eac-8573-24c959dba998 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:05.122470 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd002f2d-70ea-4eac-8573-24c959dba998 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:05.139830 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-bd002f2d-70ea-4eac-8573-24c959dba998 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:10:05.141127 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-bd002f2d-70ea-4eac-8573-24c959dba998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:10:05.184282 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:10:05.205063 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1176/2372] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:10:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:10:05.215000 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cfb570-a8e6-4bfb-9137-8b39aad8a950 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:10:05.216462 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-50cfb570-a8e6-4bfb-9137-8b39aad8a950 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:10:05.280359 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cfb570-a8e6-4bfb-9137-8b39aad8a950 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:10:05.280359 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce119ae8-5f1e-43db-b7e0-76946b8af64f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:10:05.280983 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce119ae8-5f1e-43db-b7e0-76946b8af64f service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:05.281269 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce119ae8-5f1e-43db-b7e0-76946b8af64f service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:05.281553 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce119ae8-5f1e-43db-b7e0-76946b8af64f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:05.291023 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce119ae8-5f1e-43db-b7e0-76946b8af64f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:10:05.292489 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce119ae8-5f1e-43db-b7e0-76946b8af64f service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:10:05.303137 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1197/2373] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:10:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:10:05.715640 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd002f2d-70ea-4eac-8573-24c959dba998 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:10:05.717132 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-bd002f2d-70ea-4eac-8573-24c959dba998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:10:05.729651 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd002f2d-70ea-4eac-8573-24c959dba998 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:10:05.734526 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dab7b40-1b7d-4142-a44a-b325bdd4df95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:10:05.735395 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dab7b40-1b7d-4142-a44a-b325bdd4df95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:05.735681 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dab7b40-1b7d-4142-a44a-b325bdd4df95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:05.735994 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dab7b40-1b7d-4142-a44a-b325bdd4df95 admin admin] PATH_INFO: `/v3/projects/d002c6244aa448ebb36438ad64585995` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:05.737203 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dab7b40-1b7d-4142-a44a-b325bdd4df95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d002c6244aa448ebb36438ad64585995)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:10:05.739127 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dab7b40-1b7d-4142-a44a-b325bdd4df95 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:10:05.845611 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4dab7b40-1b7d-4142-a44a-b325bdd4df95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd002c6244aa448ebb36438ad64585995', 'request_id': 'req-4dab7b40-1b7d-4142-a44a-b325bdd4df95'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:05.851112 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4dab7b40-1b7d-4142-a44a-b325bdd4df95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd002c6244aa448ebb36438ad64585995', 'request_id': 'req-4dab7b40-1b7d-4142-a44a-b325bdd4df95'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:05.857532 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4dab7b40-1b7d-4142-a44a-b325bdd4df95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd002c6244aa448ebb36438ad64585995', 'request_id': 'req-4dab7b40-1b7d-4142-a44a-b325bdd4df95'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:05.918859 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4dab7b40-1b7d-4142-a44a-b325bdd4df95 admin admin] The token cache is being invalidate because project d002c6244aa448ebb36438ad64585995 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:10:05.919156 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4dab7b40-1b7d-4142-a44a-b325bdd4df95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:05.919777 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4dab7b40-1b7d-4142-a44a-b325bdd4df95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:05.921374 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1177/2374] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:10:05 2022] DELETE /identity/v3/projects/d002c6244aa448ebb36438ad64585995 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:10:05.925700 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce119ae8-5f1e-43db-b7e0-76946b8af64f service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:10:05.935657 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce119ae8-5f1e-43db-b7e0-76946b8af64f service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:10:05.991929 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813014fa-4660-4eb8-b0a2-b605417ad8cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:10:05.992806 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813014fa-4660-4eb8-b0a2-b605417ad8cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:05.993092 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813014fa-4660-4eb8-b0a2-b605417ad8cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:05.993370 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813014fa-4660-4eb8-b0a2-b605417ad8cb admin admin] PATH_INFO: `/v3/users/47cd8fd04e9e4d01baf3e455ff836327` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:05.994564 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813014fa-4660-4eb8-b0a2-b605417ad8cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=47cd8fd04e9e4d01baf3e455ff836327)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:10:05.996527 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813014fa-4660-4eb8-b0a2-b605417ad8cb admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:10:06.200837 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:10:06.203375 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-813014fa-4660-4eb8-b0a2-b605417ad8cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47cd8fd04e9e4d01baf3e455ff836327'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:06.213940 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-813014fa-4660-4eb8-b0a2-b605417ad8cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47cd8fd04e9e4d01baf3e455ff836327'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:06.317287 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-813014fa-4660-4eb8-b0a2-b605417ad8cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47cd8fd04e9e4d01baf3e455ff836327'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:06.318110 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-813014fa-4660-4eb8-b0a2-b605417ad8cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47cd8fd04e9e4d01baf3e455ff836327'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:06.333831 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-813014fa-4660-4eb8-b0a2-b605417ad8cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47cd8fd04e9e4d01baf3e455ff836327'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:06.348126 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1198/2375] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:10:05 2022] DELETE /identity/v3/users/47cd8fd04e9e4d01baf3e455ff836327 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:10:06.816960 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dab7b40-1b7d-4142-a44a-b325bdd4df95 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:10:06.816960 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-4dab7b40-1b7d-4142-a44a-b325bdd4df95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:10:06.825277 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dab7b40-1b7d-4142-a44a-b325bdd4df95 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:10:06.898288 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07838f11-19cc-43d1-ae14-830ade098077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:10:06.899263 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07838f11-19cc-43d1-ae14-830ade098077 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:06.899569 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07838f11-19cc-43d1-ae14-830ade098077 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:06.899853 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07838f11-19cc-43d1-ae14-830ade098077 admin admin] PATH_INFO: `/v3/projects/efe70c22fd4b40c2a2c5a10e31ece4c4` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:06.901096 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07838f11-19cc-43d1-ae14-830ade098077 admin admin] RBAC: Authorizing `identity:delete_project(project_id=efe70c22fd4b40c2a2c5a10e31ece4c4)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:10:06.903346 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07838f11-19cc-43d1-ae14-830ade098077 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:10:07.035148 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-07838f11-19cc-43d1-ae14-830ade098077 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'efe70c22fd4b40c2a2c5a10e31ece4c4', 'request_id': 'req-07838f11-19cc-43d1-ae14-830ade098077'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:07.040456 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-07838f11-19cc-43d1-ae14-830ade098077 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'efe70c22fd4b40c2a2c5a10e31ece4c4', 'request_id': 'req-07838f11-19cc-43d1-ae14-830ade098077'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:07.053400 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-07838f11-19cc-43d1-ae14-830ade098077 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'efe70c22fd4b40c2a2c5a10e31ece4c4', 'request_id': 'req-07838f11-19cc-43d1-ae14-830ade098077'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:07.133556 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:10:07.141469 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-07838f11-19cc-43d1-ae14-830ade098077 admin admin] The token cache is being invalidate because project efe70c22fd4b40c2a2c5a10e31ece4c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:10:07.141469 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-07838f11-19cc-43d1-ae14-830ade098077 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:07.141469 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-07838f11-19cc-43d1-ae14-830ade098077 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:07.141469 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1178/2376] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:10:06 2022] DELETE /identity/v3/projects/efe70c22fd4b40c2a2c5a10e31ece4c4 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:10:07.147127 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fee422-5251-418c-b13c-a4ae03674e35 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:07.148301 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fee422-5251-418c-b13c-a4ae03674e35 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:07.148934 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fee422-5251-418c-b13c-a4ae03674e35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:07.169412 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-30fee422-5251-418c-b13c-a4ae03674e35 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:10:07.170774 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-30fee422-5251-418c-b13c-a4ae03674e35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:10:07.234466 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1199/2377] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:10:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:10:07.243587 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5efda36-6973-47e2-930e-9894ffc5f816 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:07.243887 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5efda36-6973-47e2-930e-9894ffc5f816 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:07.244182 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5efda36-6973-47e2-930e-9894ffc5f816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:07.263501 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-f5efda36-6973-47e2-930e-9894ffc5f816 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:10:07.264744 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f5efda36-6973-47e2-930e-9894ffc5f816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:10:07.277591 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1179/2378] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:10:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:10:07.282886 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fee422-5251-418c-b13c-a4ae03674e35 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:10:07.292929 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fee422-5251-418c-b13c-a4ae03674e35 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:10:07.298102 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20900c7-eb14-4338-a38e-e5ef042595a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:10:07.301783 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20900c7-eb14-4338-a38e-e5ef042595a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:07.301783 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20900c7-eb14-4338-a38e-e5ef042595a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:07.301783 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20900c7-eb14-4338-a38e-e5ef042595a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:07.301783 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20900c7-eb14-4338-a38e-e5ef042595a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 13:10:07.301783 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20900c7-eb14-4338-a38e-e5ef042595a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:10:07.303328 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20900c7-eb14-4338-a38e-e5ef042595a4 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:10:07.316810 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1200/2379] 10.222.0.27 () {58 vars in 1152 bytes} [Fri Jun 10 13:10:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 13:10:07.322108 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5efda36-6973-47e2-930e-9894ffc5f816 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:10:07.333347 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5efda36-6973-47e2-930e-9894ffc5f816 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:10:07.338109 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1490802b-facf-4a77-871e-171cffd82b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:10:07.338781 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1490802b-facf-4a77-871e-171cffd82b95 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:07.339070 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1490802b-facf-4a77-871e-171cffd82b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:07.344724 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1490802b-facf-4a77-871e-171cffd82b95 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:07.344724 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1490802b-facf-4a77-871e-171cffd82b95 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:10:07.344724 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1490802b-facf-4a77-871e-171cffd82b95 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:10:07.431904 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1180/2380] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:10:07 2022] POST /identity/v3/projects => generated 377 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:10:07.435052 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20900c7-eb14-4338-a38e-e5ef042595a4 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:10:07.446749 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20900c7-eb14-4338-a38e-e5ef042595a4 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:10:07.519984 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7c832b-5320-4c32-b5ea-4499721e38b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:10:07.520698 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7c832b-5320-4c32-b5ea-4499721e38b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:07.521111 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7c832b-5320-4c32-b5ea-4499721e38b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:07.521392 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7c832b-5320-4c32-b5ea-4499721e38b3 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:07.522131 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7c832b-5320-4c32-b5ea-4499721e38b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:10:07.524071 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7c832b-5320-4c32-b5ea-4499721e38b3 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:10:07.628508 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1201/2381] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:10:07 2022] POST /identity/v3/users => generated 289 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:10:07.629714 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1490802b-facf-4a77-871e-171cffd82b95 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:10:07.642854 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1490802b-facf-4a77-871e-171cffd82b95 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:10:07.647957 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bded4e9-2ac0-4092-a2a1-0d9990bcf298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:10:07.648724 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bded4e9-2ac0-4092-a2a1-0d9990bcf298 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:07.649014 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bded4e9-2ac0-4092-a2a1-0d9990bcf298 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:07.649299 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bded4e9-2ac0-4092-a2a1-0d9990bcf298 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:07.650241 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bded4e9-2ac0-4092-a2a1-0d9990bcf298 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:10:07.652625 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bded4e9-2ac0-4092-a2a1-0d9990bcf298 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:10:07.668391 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1181/2382] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:10:07 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:10:07.673048 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7c832b-5320-4c32-b5ea-4499721e38b3 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:10:07.686775 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7c832b-5320-4c32-b5ea-4499721e38b3 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:10:07.692942 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b917fd-1193-4dc5-be05-4b96f7e7a7de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:10:07.693716 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b917fd-1193-4dc5-be05-4b96f7e7a7de admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:07.694071 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b917fd-1193-4dc5-be05-4b96f7e7a7de admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:07.694479 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b917fd-1193-4dc5-be05-4b96f7e7a7de admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:07.695277 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b917fd-1193-4dc5-be05-4b96f7e7a7de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:10:07.698355 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b917fd-1193-4dc5-be05-4b96f7e7a7de admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:10:07.716461 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1202/2383] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:10:07 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 10 13:10:07.717583 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bded4e9-2ac0-4092-a2a1-0d9990bcf298 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:10:07.732626 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bded4e9-2ac0-4092-a2a1-0d9990bcf298 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:10:07.738258 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd0c8cb-0137-4181-be8a-783102bfa5ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:10:07.739144 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd0c8cb-0137-4181-be8a-783102bfa5ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:07.739422 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd0c8cb-0137-4181-be8a-783102bfa5ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:07.739702 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd0c8cb-0137-4181-be8a-783102bfa5ab admin admin] PATH_INFO: `/v3/projects/6e6a747741724de0acb1bd53b6ca3ed3/users/46e816f7942443d5a2b1111b0825b4ac/roles/43106c664e1b40de984c0c62701149c8` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:07.755760 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd0c8cb-0137-4181-be8a-783102bfa5ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e6a747741724de0acb1bd53b6ca3ed3, user_id=46e816f7942443d5a2b1111b0825b4ac, role_id=43106c664e1b40de984c0c62701149c8)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:10:07.758937 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd0c8cb-0137-4181-be8a-783102bfa5ab admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:10:07.811656 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1182/2384] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:10:07 2022] PUT /identity/v3/projects/6e6a747741724de0acb1bd53b6ca3ed3/users/46e816f7942443d5a2b1111b0825b4ac/roles/43106c664e1b40de984c0c62701149c8 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 10 13:10:07.822582 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7ac1d7-bcf0-44e1-a589-41c2eac12cd7 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:07.822885 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7ac1d7-bcf0-44e1-a589-41c2eac12cd7 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:07.823198 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7ac1d7-bcf0-44e1-a589-41c2eac12cd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:07.864641 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-1d7ac1d7-bcf0-44e1-a589-41c2eac12cd7 None None] MFA Rules not processed for user `46e816f7942443d5a2b1111b0825b4ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:10:07.866082 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1d7ac1d7-bcf0-44e1-a589-41c2eac12cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:10:07.897638 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:10:08.030286 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1203/2385] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:10:07 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:10:08.031554 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd0c8cb-0137-4181-be8a-783102bfa5ab admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:10:08.128752 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd0c8cb-0137-4181-be8a-783102bfa5ab admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:10:08.139107 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bdd377-fa4c-4fe5-8e76-5e6a52472067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:10:08.139976 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bdd377-fa4c-4fe5-8e76-5e6a52472067 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:08.140397 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bdd377-fa4c-4fe5-8e76-5e6a52472067 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:08.141474 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bdd377-fa4c-4fe5-8e76-5e6a52472067 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:08.146047 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bdd377-fa4c-4fe5-8e76-5e6a52472067 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:10:08.148561 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bdd377-fa4c-4fe5-8e76-5e6a52472067 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:10:08.259765 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1183/2386] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:10:08 2022] POST /identity/v3/projects => generated 379 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:10:08.265125 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7ac1d7-bcf0-44e1-a589-41c2eac12cd7 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:10:08.278091 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7ac1d7-bcf0-44e1-a589-41c2eac12cd7 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:10:08.346622 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365f1517-6616-42f8-834c-26d2b799edf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:10:08.347196 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365f1517-6616-42f8-834c-26d2b799edf3 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:08.347476 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365f1517-6616-42f8-834c-26d2b799edf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:08.347748 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365f1517-6616-42f8-834c-26d2b799edf3 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:08.348607 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365f1517-6616-42f8-834c-26d2b799edf3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:10:08.350504 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365f1517-6616-42f8-834c-26d2b799edf3 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:10:08.437103 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1204/2387] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:10:08 2022] POST /identity/v3/users => generated 290 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:10:08.437264 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bdd377-fa4c-4fe5-8e76-5e6a52472067 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:10:08.447093 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bdd377-fa4c-4fe5-8e76-5e6a52472067 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:10:08.453958 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb40e91b-b64c-49d1-8030-305123f53271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:10:08.456425 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb40e91b-b64c-49d1-8030-305123f53271 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:08.456856 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb40e91b-b64c-49d1-8030-305123f53271 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:08.457148 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb40e91b-b64c-49d1-8030-305123f53271 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:08.457901 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb40e91b-b64c-49d1-8030-305123f53271 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:10:08.459681 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb40e91b-b64c-49d1-8030-305123f53271 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:10:08.475563 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1184/2388] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:10:08 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 10 13:10:08.479238 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365f1517-6616-42f8-834c-26d2b799edf3 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:10:08.490572 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365f1517-6616-42f8-834c-26d2b799edf3 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:10:08.495941 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15edbf2d-0818-4ca0-8029-5d86354f5adb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:10:08.495941 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15edbf2d-0818-4ca0-8029-5d86354f5adb admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:08.496183 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15edbf2d-0818-4ca0-8029-5d86354f5adb admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:08.496414 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15edbf2d-0818-4ca0-8029-5d86354f5adb admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:08.497209 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15edbf2d-0818-4ca0-8029-5d86354f5adb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:10:08.498798 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15edbf2d-0818-4ca0-8029-5d86354f5adb admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:10:08.510106 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1205/2389] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:10:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:10:08.514307 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb40e91b-b64c-49d1-8030-305123f53271 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:10:08.525016 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb40e91b-b64c-49d1-8030-305123f53271 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:10:08.530291 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89070647-5ad3-4ead-9029-c35ed9cba554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:10:08.531298 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89070647-5ad3-4ead-9029-c35ed9cba554 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:08.531644 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89070647-5ad3-4ead-9029-c35ed9cba554 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:08.532016 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89070647-5ad3-4ead-9029-c35ed9cba554 admin admin] PATH_INFO: `/v3/projects/98b6f5531bca4ee3b75e93766acfafc8/users/1f290e0c239446bf8e2948445b59134f/roles/43106c664e1b40de984c0c62701149c8` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:08.549740 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89070647-5ad3-4ead-9029-c35ed9cba554 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98b6f5531bca4ee3b75e93766acfafc8, user_id=1f290e0c239446bf8e2948445b59134f, role_id=43106c664e1b40de984c0c62701149c8)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:10:08.552590 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89070647-5ad3-4ead-9029-c35ed9cba554 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:10:08.612321 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1185/2390] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:10:08 2022] PUT /identity/v3/projects/98b6f5531bca4ee3b75e93766acfafc8/users/1f290e0c239446bf8e2948445b59134f/roles/43106c664e1b40de984c0c62701149c8 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:10:08.623607 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4217d0-2115-40e3-9a3f-a62430e5f4d0 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:08.623942 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4217d0-2115-40e3-9a3f-a62430e5f4d0 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:08.624231 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4217d0-2115-40e3-9a3f-a62430e5f4d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:08.653804 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-ec4217d0-2115-40e3-9a3f-a62430e5f4d0 None None] MFA Rules not processed for user `1f290e0c239446bf8e2948445b59134f`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:10:08.655023 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ec4217d0-2115-40e3-9a3f-a62430e5f4d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:10:08.757325 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1206/2391] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:10:08 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:10:08.768649 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89070647-5ad3-4ead-9029-c35ed9cba554 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:10:08.846021 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:10:08.870597 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89070647-5ad3-4ead-9029-c35ed9cba554 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:10:08.879209 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5b49cd-39b5-4714-af6b-f53b36d3a993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:10:08.881255 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5b49cd-39b5-4714-af6b-f53b36d3a993 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:08.881255 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5b49cd-39b5-4714-af6b-f53b36d3a993 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:08.881255 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5b49cd-39b5-4714-af6b-f53b36d3a993 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:08.884135 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5b49cd-39b5-4714-af6b-f53b36d3a993 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:10:08.884646 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5b49cd-39b5-4714-af6b-f53b36d3a993 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:10:08.975898 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1186/2392] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:10:08 2022] POST /identity/v3/projects => generated 377 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 10 13:10:08.978075 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4217d0-2115-40e3-9a3f-a62430e5f4d0 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:10:08.989010 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4217d0-2115-40e3-9a3f-a62430e5f4d0 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:10:09.012496 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:10:09.050406 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1194f6c1-645b-4459-9dbd-9b0e748e822e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:10:09.051085 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1194f6c1-645b-4459-9dbd-9b0e748e822e admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:09.051366 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1194f6c1-645b-4459-9dbd-9b0e748e822e admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:09.051642 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1194f6c1-645b-4459-9dbd-9b0e748e822e admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:09.052375 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1194f6c1-645b-4459-9dbd-9b0e748e822e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:10:09.054276 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1194f6c1-645b-4459-9dbd-9b0e748e822e admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:10:09.153428 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1207/2393] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:10:08 2022] POST /identity/v3/users => generated 295 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 10 13:10:09.154512 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5b49cd-39b5-4714-af6b-f53b36d3a993 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:10:09.165237 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5b49cd-39b5-4714-af6b-f53b36d3a993 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:10:09.169841 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8877898f-a201-4f72-9e35-30995e59cd11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:10:09.170510 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8877898f-a201-4f72-9e35-30995e59cd11 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:09.170801 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8877898f-a201-4f72-9e35-30995e59cd11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:09.171081 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8877898f-a201-4f72-9e35-30995e59cd11 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:09.171811 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8877898f-a201-4f72-9e35-30995e59cd11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:10:09.173587 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8877898f-a201-4f72-9e35-30995e59cd11 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:10:09.186180 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1187/2394] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:10:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 10 13:10:09.190776 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1194f6c1-645b-4459-9dbd-9b0e748e822e admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:10:09.200757 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1194f6c1-645b-4459-9dbd-9b0e748e822e admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:10:09.205420 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074e3a99-27d2-4b91-bd10-0660f7b149d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:10:09.206330 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074e3a99-27d2-4b91-bd10-0660f7b149d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:09.212910 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074e3a99-27d2-4b91-bd10-0660f7b149d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:09.212910 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074e3a99-27d2-4b91-bd10-0660f7b149d6 admin admin] PATH_INFO: `/v3/projects/f630cd27fc744d3887cd940e32d8c49b/users/1fda3254a90a410aa6dd02e4dba64f55/roles/b43d5a8247e244869c2c70f986f42e8a` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:09.221685 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074e3a99-27d2-4b91-bd10-0660f7b149d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f630cd27fc744d3887cd940e32d8c49b, user_id=1fda3254a90a410aa6dd02e4dba64f55, role_id=b43d5a8247e244869c2c70f986f42e8a)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:10:09.224344 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074e3a99-27d2-4b91-bd10-0660f7b149d6 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:10:09.308261 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1208/2395] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:10:09 2022] PUT /identity/v3/projects/f630cd27fc744d3887cd940e32d8c49b/users/1fda3254a90a410aa6dd02e4dba64f55/roles/b43d5a8247e244869c2c70f986f42e8a => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 10 13:10:09.312204 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8877898f-a201-4f72-9e35-30995e59cd11 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:10:09.323212 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8877898f-a201-4f72-9e35-30995e59cd11 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:10:09.397474 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904cbb53-b9b4-49e9-a2fb-cffb076faefe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:10:09.398151 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904cbb53-b9b4-49e9-a2fb-cffb076faefe admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:09.398441 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904cbb53-b9b4-49e9-a2fb-cffb076faefe admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:09.398729 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904cbb53-b9b4-49e9-a2fb-cffb076faefe admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:09.399502 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904cbb53-b9b4-49e9-a2fb-cffb076faefe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:10:09.401335 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904cbb53-b9b4-49e9-a2fb-cffb076faefe admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:10:09.412553 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1188/2396] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:10:09 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:10:09.422125 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074e3a99-27d2-4b91-bd10-0660f7b149d6 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:10:09.433395 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074e3a99-27d2-4b91-bd10-0660f7b149d6 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:10:09.438423 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d08a8c-970f-49ad-a984-e50db8b4c04e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:10:09.439011 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d08a8c-970f-49ad-a984-e50db8b4c04e admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:09.439296 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d08a8c-970f-49ad-a984-e50db8b4c04e admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:09.439577 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d08a8c-970f-49ad-a984-e50db8b4c04e admin admin] PATH_INFO: `/v3/projects/f630cd27fc744d3887cd940e32d8c49b/users/1fda3254a90a410aa6dd02e4dba64f55/roles/b43d5a8247e244869c2c70f986f42e8a` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:09.442249 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d08a8c-970f-49ad-a984-e50db8b4c04e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f630cd27fc744d3887cd940e32d8c49b, user_id=1fda3254a90a410aa6dd02e4dba64f55, role_id=b43d5a8247e244869c2c70f986f42e8a)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:10:09.444933 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d08a8c-970f-49ad-a984-e50db8b4c04e admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:10:09.456191 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1209/2397] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:10:09 2022] PUT /identity/v3/projects/f630cd27fc744d3887cd940e32d8c49b/users/1fda3254a90a410aa6dd02e4dba64f55/roles/b43d5a8247e244869c2c70f986f42e8a => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:10:09.466286 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82531c0d-24b0-43ac-93cd-e76d8cbf0344 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:09.466640 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82531c0d-24b0-43ac-93cd-e76d8cbf0344 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:09.466920 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82531c0d-24b0-43ac-93cd-e76d8cbf0344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:09.495541 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-82531c0d-24b0-43ac-93cd-e76d8cbf0344 None None] MFA Rules not processed for user `1fda3254a90a410aa6dd02e4dba64f55`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:10:09.496843 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-82531c0d-24b0-43ac-93cd-e76d8cbf0344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:10:09.570597 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1189/2398] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:10:09 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:10:09.582322 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d08a8c-970f-49ad-a984-e50db8b4c04e admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:10:09.584041 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-63d08a8c-970f-49ad-a984-e50db8b4c04e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:10:09.645261 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d08a8c-970f-49ad-a984-e50db8b4c04e admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:10:09.648982 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948eed15-9846-40c8-a385-b4f453d94572 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6cc95a563b941ac92ab283aa0d5a5d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:10:09.649770 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948eed15-9846-40c8-a385-b4f453d94572 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:09.649972 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948eed15-9846-40c8-a385-b4f453d94572 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:09.650295 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948eed15-9846-40c8-a385-b4f453d94572 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:09.660205 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948eed15-9846-40c8-a385-b4f453d94572 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:10:09.661632 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948eed15-9846-40c8-a385-b4f453d94572 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:10:09.671549 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1210/2399] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:10:09 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:10:09.770388 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82531c0d-24b0-43ac-93cd-e76d8cbf0344 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:10:09.771832 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-82531c0d-24b0-43ac-93cd-e76d8cbf0344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:10:09.784526 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82531c0d-24b0-43ac-93cd-e76d8cbf0344 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:10:09.788459 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117312bb-3321-4c81-abff-38781a904d1f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6cc95a563b941ac92ab283aa0d5a5d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:10:09.790095 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117312bb-3321-4c81-abff-38781a904d1f service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:09.790095 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117312bb-3321-4c81-abff-38781a904d1f service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:09.790095 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117312bb-3321-4c81-abff-38781a904d1f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:09.834487 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:10:09.851609 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117312bb-3321-4c81-abff-38781a904d1f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:10:09.856547 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117312bb-3321-4c81-abff-38781a904d1f service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:10:09.865612 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1190/2400] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:10:09 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:10:13.647511 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948eed15-9846-40c8-a385-b4f453d94572 service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:10:13.711701 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948eed15-9846-40c8-a385-b4f453d94572 service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:10:13.716119 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77350e4-467b-4cef-a0cc-ba0c265fd698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:10:13.716986 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77350e4-467b-4cef-a0cc-ba0c265fd698 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:13.717268 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77350e4-467b-4cef-a0cc-ba0c265fd698 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:13.717544 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77350e4-467b-4cef-a0cc-ba0c265fd698 admin admin] PATH_INFO: `/v3/users/46e816f7942443d5a2b1111b0825b4ac` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:13.718704 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77350e4-467b-4cef-a0cc-ba0c265fd698 admin admin] RBAC: Authorizing `identity:delete_user(user_id=46e816f7942443d5a2b1111b0825b4ac)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:10:13.720648 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77350e4-467b-4cef-a0cc-ba0c265fd698 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:10:13.742028 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:10:13.930540 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f77350e4-467b-4cef-a0cc-ba0c265fd698 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46e816f7942443d5a2b1111b0825b4ac'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:13.937919 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f77350e4-467b-4cef-a0cc-ba0c265fd698 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46e816f7942443d5a2b1111b0825b4ac'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:13.999912 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f77350e4-467b-4cef-a0cc-ba0c265fd698 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46e816f7942443d5a2b1111b0825b4ac'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:14.000545 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f77350e4-467b-4cef-a0cc-ba0c265fd698 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46e816f7942443d5a2b1111b0825b4ac'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:14.013931 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f77350e4-467b-4cef-a0cc-ba0c265fd698 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46e816f7942443d5a2b1111b0825b4ac'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:14.021022 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1211/2401] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:10:13 2022] DELETE /identity/v3/users/46e816f7942443d5a2b1111b0825b4ac => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:10:14.026168 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b1cc86-0c99-4e09-831b-dda5f8250d47 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:14.026677 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b1cc86-0c99-4e09-831b-dda5f8250d47 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:14.026797 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b1cc86-0c99-4e09-831b-dda5f8250d47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:14.045136 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-44b1cc86-0c99-4e09-831b-dda5f8250d47 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:10:14.046555 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-44b1cc86-0c99-4e09-831b-dda5f8250d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:10:14.117324 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1191/2402] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:10:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:10:14.126990 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77350e4-467b-4cef-a0cc-ba0c265fd698 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:10:14.129185 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f77350e4-467b-4cef-a0cc-ba0c265fd698 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:10:14.197041 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77350e4-467b-4cef-a0cc-ba0c265fd698 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:10:14.201211 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2de4183-2bae-441d-bf6e-dcfdca79ba17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:10:14.201857 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2de4183-2bae-441d-bf6e-dcfdca79ba17 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:14.202139 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2de4183-2bae-441d-bf6e-dcfdca79ba17 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:14.202416 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2de4183-2bae-441d-bf6e-dcfdca79ba17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:14.213256 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2de4183-2bae-441d-bf6e-dcfdca79ba17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:10:14.219358 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2de4183-2bae-441d-bf6e-dcfdca79ba17 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:10:14.225935 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1212/2403] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:10:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:10:14.664071 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b1cc86-0c99-4e09-831b-dda5f8250d47 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:10:14.665503 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-44b1cc86-0c99-4e09-831b-dda5f8250d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:10:14.678484 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b1cc86-0c99-4e09-831b-dda5f8250d47 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:10:14.683227 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9393cec0-5c0f-4126-a2a3-10c401dfc792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:10:14.684727 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9393cec0-5c0f-4126-a2a3-10c401dfc792 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:14.685289 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9393cec0-5c0f-4126-a2a3-10c401dfc792 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:14.685731 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9393cec0-5c0f-4126-a2a3-10c401dfc792 admin admin] PATH_INFO: `/v3/projects/6e6a747741724de0acb1bd53b6ca3ed3` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:14.687050 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9393cec0-5c0f-4126-a2a3-10c401dfc792 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e6a747741724de0acb1bd53b6ca3ed3)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:10:14.689007 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9393cec0-5c0f-4126-a2a3-10c401dfc792 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:10:14.794168 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-9393cec0-5c0f-4126-a2a3-10c401dfc792 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e6a747741724de0acb1bd53b6ca3ed3', 'request_id': 'req-9393cec0-5c0f-4126-a2a3-10c401dfc792'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:14.800126 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-9393cec0-5c0f-4126-a2a3-10c401dfc792 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e6a747741724de0acb1bd53b6ca3ed3', 'request_id': 'req-9393cec0-5c0f-4126-a2a3-10c401dfc792'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:14.806127 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-9393cec0-5c0f-4126-a2a3-10c401dfc792 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e6a747741724de0acb1bd53b6ca3ed3', 'request_id': 'req-9393cec0-5c0f-4126-a2a3-10c401dfc792'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:14.884853 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-9393cec0-5c0f-4126-a2a3-10c401dfc792 admin admin] The token cache is being invalidate because project 6e6a747741724de0acb1bd53b6ca3ed3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:10:14.885145 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-9393cec0-5c0f-4126-a2a3-10c401dfc792 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:14.885779 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-9393cec0-5c0f-4126-a2a3-10c401dfc792 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:14.887328 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1192/2404] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:10:14 2022] DELETE /identity/v3/projects/6e6a747741724de0acb1bd53b6ca3ed3 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:10:14.891427 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2de4183-2bae-441d-bf6e-dcfdca79ba17 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:10:14.900863 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2de4183-2bae-441d-bf6e-dcfdca79ba17 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:10:14.957382 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efda9cc3-9e4e-438c-a26d-e27d511ff1a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:10:14.958272 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efda9cc3-9e4e-438c-a26d-e27d511ff1a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:14.958554 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efda9cc3-9e4e-438c-a26d-e27d511ff1a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:14.958828 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efda9cc3-9e4e-438c-a26d-e27d511ff1a4 admin admin] PATH_INFO: `/v3/users/1f290e0c239446bf8e2948445b59134f` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:14.960030 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efda9cc3-9e4e-438c-a26d-e27d511ff1a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f290e0c239446bf8e2948445b59134f)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:10:14.961926 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efda9cc3-9e4e-438c-a26d-e27d511ff1a4 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:10:15.117363 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:10:15.120896 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-efda9cc3-9e4e-438c-a26d-e27d511ff1a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f290e0c239446bf8e2948445b59134f'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:15.132027 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-efda9cc3-9e4e-438c-a26d-e27d511ff1a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f290e0c239446bf8e2948445b59134f'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:15.251035 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-efda9cc3-9e4e-438c-a26d-e27d511ff1a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f290e0c239446bf8e2948445b59134f'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:15.251647 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-efda9cc3-9e4e-438c-a26d-e27d511ff1a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f290e0c239446bf8e2948445b59134f'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:15.263853 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-efda9cc3-9e4e-438c-a26d-e27d511ff1a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f290e0c239446bf8e2948445b59134f'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:15.270386 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1213/2405] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:10:14 2022] DELETE /identity/v3/users/1f290e0c239446bf8e2948445b59134f => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:10:15.647538 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9393cec0-5c0f-4126-a2a3-10c401dfc792 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:10:15.649131 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-9393cec0-5c0f-4126-a2a3-10c401dfc792 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:10:15.663475 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9393cec0-5c0f-4126-a2a3-10c401dfc792 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:10:15.723335 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0414f4-a66f-4f71-a6f4-41a120761837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:10:15.724208 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0414f4-a66f-4f71-a6f4-41a120761837 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:15.724487 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0414f4-a66f-4f71-a6f4-41a120761837 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:15.724775 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0414f4-a66f-4f71-a6f4-41a120761837 admin admin] PATH_INFO: `/v3/projects/98b6f5531bca4ee3b75e93766acfafc8` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:15.725983 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0414f4-a66f-4f71-a6f4-41a120761837 admin admin] RBAC: Authorizing `identity:delete_project(project_id=98b6f5531bca4ee3b75e93766acfafc8)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:10:15.727823 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0414f4-a66f-4f71-a6f4-41a120761837 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:10:15.734095 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:10:15.862872 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6b0414f4-a66f-4f71-a6f4-41a120761837 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98b6f5531bca4ee3b75e93766acfafc8', 'request_id': 'req-6b0414f4-a66f-4f71-a6f4-41a120761837'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:15.868621 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6b0414f4-a66f-4f71-a6f4-41a120761837 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98b6f5531bca4ee3b75e93766acfafc8', 'request_id': 'req-6b0414f4-a66f-4f71-a6f4-41a120761837'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:15.874784 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6b0414f4-a66f-4f71-a6f4-41a120761837 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98b6f5531bca4ee3b75e93766acfafc8', 'request_id': 'req-6b0414f4-a66f-4f71-a6f4-41a120761837'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:15.970011 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6b0414f4-a66f-4f71-a6f4-41a120761837 admin admin] The token cache is being invalidate because project 98b6f5531bca4ee3b75e93766acfafc8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:10:15.970292 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6b0414f4-a66f-4f71-a6f4-41a120761837 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:15.971006 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6b0414f4-a66f-4f71-a6f4-41a120761837 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:15.972630 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1193/2406] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:10:15 2022] DELETE /identity/v3/projects/98b6f5531bca4ee3b75e93766acfafc8 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:10:15.976830 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efda9cc3-9e4e-438c-a26d-e27d511ff1a4 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:10:15.978157 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-efda9cc3-9e4e-438c-a26d-e27d511ff1a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:10:15.991192 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efda9cc3-9e4e-438c-a26d-e27d511ff1a4 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:10:16.049900 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3b463a-fec2-446e-904c-994879f971e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:10:16.050771 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3b463a-fec2-446e-904c-994879f971e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:16.051056 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3b463a-fec2-446e-904c-994879f971e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:16.051334 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3b463a-fec2-446e-904c-994879f971e8 admin admin] PATH_INFO: `/v3/users/1fda3254a90a410aa6dd02e4dba64f55` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:16.052553 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3b463a-fec2-446e-904c-994879f971e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1fda3254a90a410aa6dd02e4dba64f55)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:10:16.054459 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3b463a-fec2-446e-904c-994879f971e8 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:10:16.219137 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4a3b463a-fec2-446e-904c-994879f971e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1fda3254a90a410aa6dd02e4dba64f55'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:16.226474 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4a3b463a-fec2-446e-904c-994879f971e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1fda3254a90a410aa6dd02e4dba64f55'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:16.279578 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4a3b463a-fec2-446e-904c-994879f971e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1fda3254a90a410aa6dd02e4dba64f55'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:16.280314 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4a3b463a-fec2-446e-904c-994879f971e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1fda3254a90a410aa6dd02e4dba64f55'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:16.294917 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4a3b463a-fec2-446e-904c-994879f971e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1fda3254a90a410aa6dd02e4dba64f55'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:16.301816 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1214/2407] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:10:15 2022] DELETE /identity/v3/users/1fda3254a90a410aa6dd02e4dba64f55 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:10:16.906175 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0414f4-a66f-4f71-a6f4-41a120761837 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:10:16.909069 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-6b0414f4-a66f-4f71-a6f4-41a120761837 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:10:16.925706 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0414f4-a66f-4f71-a6f4-41a120761837 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:10:16.994047 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3762ac0-415e-4cdb-b377-454dd7176e47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:10:16.994911 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3762ac0-415e-4cdb-b377-454dd7176e47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:16.995197 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3762ac0-415e-4cdb-b377-454dd7176e47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:16.995475 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3762ac0-415e-4cdb-b377-454dd7176e47 admin admin] PATH_INFO: `/v3/projects/f630cd27fc744d3887cd940e32d8c49b` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:16.996695 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3762ac0-415e-4cdb-b377-454dd7176e47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f630cd27fc744d3887cd940e32d8c49b)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:10:16.998599 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3762ac0-415e-4cdb-b377-454dd7176e47 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:10:17.147082 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-d3762ac0-415e-4cdb-b377-454dd7176e47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f630cd27fc744d3887cd940e32d8c49b', 'request_id': 'req-d3762ac0-415e-4cdb-b377-454dd7176e47'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:17.154121 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-d3762ac0-415e-4cdb-b377-454dd7176e47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f630cd27fc744d3887cd940e32d8c49b', 'request_id': 'req-d3762ac0-415e-4cdb-b377-454dd7176e47'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:17.160739 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-d3762ac0-415e-4cdb-b377-454dd7176e47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f630cd27fc744d3887cd940e32d8c49b', 'request_id': 'req-d3762ac0-415e-4cdb-b377-454dd7176e47'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:17.274206 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-d3762ac0-415e-4cdb-b377-454dd7176e47 admin admin] The token cache is being invalidate because project f630cd27fc744d3887cd940e32d8c49b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:10:17.274563 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-d3762ac0-415e-4cdb-b377-454dd7176e47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:17.275190 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-d3762ac0-415e-4cdb-b377-454dd7176e47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:17.276754 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1194/2408] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:10:16 2022] DELETE /identity/v3/projects/f630cd27fc744d3887cd940e32d8c49b => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:10:17.283183 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74854469-cb20-4f3e-9ea5-e7a9f406868c None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:17.283643 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74854469-cb20-4f3e-9ea5-e7a9f406868c None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:17.283766 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74854469-cb20-4f3e-9ea5-e7a9f406868c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:17.301228 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-74854469-cb20-4f3e-9ea5-e7a9f406868c None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:10:17.302432 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-74854469-cb20-4f3e-9ea5-e7a9f406868c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:10:17.345095 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:10:17.370142 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1215/2409] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:10:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:10:17.379363 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eefd2ff-0fdc-48de-923e-1f2cf876fd06 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:17.379674 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eefd2ff-0fdc-48de-923e-1f2cf876fd06 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:17.379979 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eefd2ff-0fdc-48de-923e-1f2cf876fd06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:17.397784 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-6eefd2ff-0fdc-48de-923e-1f2cf876fd06 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:10:17.399029 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-6eefd2ff-0fdc-48de-923e-1f2cf876fd06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:10:17.407005 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1195/2410] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:10:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:10:17.411201 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74854469-cb20-4f3e-9ea5-e7a9f406868c None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:10:17.420737 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74854469-cb20-4f3e-9ea5-e7a9f406868c None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:10:17.425682 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1abce1-4fae-48bf-a1cf-2e3482aa1fbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:10:17.426322 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1abce1-4fae-48bf-a1cf-2e3482aa1fbb admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:17.426602 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1abce1-4fae-48bf-a1cf-2e3482aa1fbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:17.426875 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1abce1-4fae-48bf-a1cf-2e3482aa1fbb admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:17.427588 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1abce1-4fae-48bf-a1cf-2e3482aa1fbb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 13:10:17.427922 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1abce1-4fae-48bf-a1cf-2e3482aa1fbb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:10:17.429487 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1abce1-4fae-48bf-a1cf-2e3482aa1fbb admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:10:17.441187 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1216/2411] 10.222.0.27 () {58 vars in 1152 bytes} [Fri Jun 10 13:10:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 13:10:17.445911 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eefd2ff-0fdc-48de-923e-1f2cf876fd06 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:10:17.455185 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eefd2ff-0fdc-48de-923e-1f2cf876fd06 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:10:17.459822 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fefdd2-ae40-4076-9582-e997b87199d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:10:17.460502 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fefdd2-ae40-4076-9582-e997b87199d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:17.460787 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fefdd2-ae40-4076-9582-e997b87199d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:17.461119 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fefdd2-ae40-4076-9582-e997b87199d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:17.461804 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fefdd2-ae40-4076-9582-e997b87199d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:10:17.463693 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fefdd2-ae40-4076-9582-e997b87199d3 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:10:17.477581 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:10:17.570638 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1196/2412] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:10:17 2022] POST /identity/v3/projects => generated 379 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:10:17.571772 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1abce1-4fae-48bf-a1cf-2e3482aa1fbb admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:10:17.581650 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1abce1-4fae-48bf-a1cf-2e3482aa1fbb admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:10:17.640389 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97b850e-20bd-45aa-adc2-03addf1ceaef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:10:17.641085 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97b850e-20bd-45aa-adc2-03addf1ceaef admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:17.641368 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97b850e-20bd-45aa-adc2-03addf1ceaef admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:17.641645 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97b850e-20bd-45aa-adc2-03addf1ceaef admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:17.642366 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97b850e-20bd-45aa-adc2-03addf1ceaef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:10:17.644378 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97b850e-20bd-45aa-adc2-03addf1ceaef admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:10:17.720917 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1217/2413] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:10:17 2022] POST /identity/v3/users => generated 290 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 10 13:10:17.725344 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fefdd2-ae40-4076-9582-e997b87199d3 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:10:17.736067 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fefdd2-ae40-4076-9582-e997b87199d3 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:10:17.740673 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480f9d96-8938-4164-b626-0f927653f3fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:10:17.741331 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480f9d96-8938-4164-b626-0f927653f3fd admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:17.741618 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480f9d96-8938-4164-b626-0f927653f3fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:17.741900 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480f9d96-8938-4164-b626-0f927653f3fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:17.742742 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480f9d96-8938-4164-b626-0f927653f3fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:10:17.744368 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480f9d96-8938-4164-b626-0f927653f3fd admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:10:17.756163 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1197/2414] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:10:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:10:17.764526 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97b850e-20bd-45aa-adc2-03addf1ceaef admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:10:17.775114 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97b850e-20bd-45aa-adc2-03addf1ceaef admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:10:17.779974 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fd7434-fc92-446b-b23b-179a94f307cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:10:17.780620 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fd7434-fc92-446b-b23b-179a94f307cd admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:17.780903 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fd7434-fc92-446b-b23b-179a94f307cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:17.781174 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fd7434-fc92-446b-b23b-179a94f307cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:17.781886 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fd7434-fc92-446b-b23b-179a94f307cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:10:17.783455 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fd7434-fc92-446b-b23b-179a94f307cd admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:10:17.793122 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1218/2415] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:10:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:10:17.797021 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480f9d96-8938-4164-b626-0f927653f3fd admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:10:17.806709 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480f9d96-8938-4164-b626-0f927653f3fd admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:10:17.811469 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173edab1-5837-488e-a608-66d22195a7c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:10:17.812356 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173edab1-5837-488e-a608-66d22195a7c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:17.812638 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173edab1-5837-488e-a608-66d22195a7c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:17.812908 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173edab1-5837-488e-a608-66d22195a7c6 admin admin] PATH_INFO: `/v3/projects/77af84c4bc7745c2ba3e8af2c63e5229/users/8cb6680f77bc43eebab12582a707b5a3/roles/43106c664e1b40de984c0c62701149c8` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:17.827504 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173edab1-5837-488e-a608-66d22195a7c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77af84c4bc7745c2ba3e8af2c63e5229, user_id=8cb6680f77bc43eebab12582a707b5a3, role_id=43106c664e1b40de984c0c62701149c8)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:10:17.830207 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173edab1-5837-488e-a608-66d22195a7c6 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:10:17.926011 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1198/2416] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:10:17 2022] PUT /identity/v3/projects/77af84c4bc7745c2ba3e8af2c63e5229/users/8cb6680f77bc43eebab12582a707b5a3/roles/43106c664e1b40de984c0c62701149c8 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 10 13:10:17.938063 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d45302-1d7c-44f3-818c-5d04093b8267 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:17.938357 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d45302-1d7c-44f3-818c-5d04093b8267 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:17.938642 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d45302-1d7c-44f3-818c-5d04093b8267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:17.967654 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-f1d45302-1d7c-44f3-818c-5d04093b8267 None None] MFA Rules not processed for user `8cb6680f77bc43eebab12582a707b5a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:10:17.968907 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f1d45302-1d7c-44f3-818c-5d04093b8267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:10:18.032068 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1219/2417] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:10:17 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:10:18.036664 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173edab1-5837-488e-a608-66d22195a7c6 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:10:18.096652 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173edab1-5837-488e-a608-66d22195a7c6 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:10:18.101262 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f06ef8-89f5-4b8e-86b3-9e18193ee9a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:10:18.101908 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f06ef8-89f5-4b8e-86b3-9e18193ee9a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:18.102193 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f06ef8-89f5-4b8e-86b3-9e18193ee9a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:18.102468 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f06ef8-89f5-4b8e-86b3-9e18193ee9a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:18.103158 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f06ef8-89f5-4b8e-86b3-9e18193ee9a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:10:18.105091 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f06ef8-89f5-4b8e-86b3-9e18193ee9a0 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:10:18.199464 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1199/2418] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:10:18 2022] POST /identity/v3/projects => generated 381 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:10:18.200850 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d45302-1d7c-44f3-818c-5d04093b8267 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:10:18.211648 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d45302-1d7c-44f3-818c-5d04093b8267 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:10:18.244281 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:10:18.269989 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f38c787-2485-4e96-b435-e4b2b7850732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:10:18.270771 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f38c787-2485-4e96-b435-e4b2b7850732 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:18.271208 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f38c787-2485-4e96-b435-e4b2b7850732 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:18.271488 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f38c787-2485-4e96-b435-e4b2b7850732 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:18.272225 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f38c787-2485-4e96-b435-e4b2b7850732 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:10:18.274145 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f38c787-2485-4e96-b435-e4b2b7850732 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:10:18.366721 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1220/2419] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:10:18 2022] POST /identity/v3/users => generated 297 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:10:18.376534 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f06ef8-89f5-4b8e-86b3-9e18193ee9a0 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:10:18.387337 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f06ef8-89f5-4b8e-86b3-9e18193ee9a0 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:10:18.391964 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d949366-e02a-4477-8c28-f491df1f1452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:10:18.392616 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d949366-e02a-4477-8c28-f491df1f1452 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:18.392898 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d949366-e02a-4477-8c28-f491df1f1452 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:18.393174 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d949366-e02a-4477-8c28-f491df1f1452 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:18.393895 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d949366-e02a-4477-8c28-f491df1f1452 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:10:18.395500 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d949366-e02a-4477-8c28-f491df1f1452 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:10:18.406262 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1200/2420] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:10:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:10:18.410137 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f38c787-2485-4e96-b435-e4b2b7850732 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:10:18.419781 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f38c787-2485-4e96-b435-e4b2b7850732 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:10:18.424776 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7f5cbc-f4f5-481b-b13b-afc10f927aa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:10:18.425636 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7f5cbc-f4f5-481b-b13b-afc10f927aa5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:18.425916 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7f5cbc-f4f5-481b-b13b-afc10f927aa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:18.426192 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7f5cbc-f4f5-481b-b13b-afc10f927aa5 admin admin] PATH_INFO: `/v3/projects/938a97d6ab4d499882c09b9ef25cc47a/users/24ac1e73cb934f11a3ef72b19f38a83c/roles/b43d5a8247e244869c2c70f986f42e8a` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:18.441216 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7f5cbc-f4f5-481b-b13b-afc10f927aa5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=938a97d6ab4d499882c09b9ef25cc47a, user_id=24ac1e73cb934f11a3ef72b19f38a83c, role_id=b43d5a8247e244869c2c70f986f42e8a)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:10:18.443725 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7f5cbc-f4f5-481b-b13b-afc10f927aa5 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:10:18.514126 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1221/2421] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:10:18 2022] PUT /identity/v3/projects/938a97d6ab4d499882c09b9ef25cc47a/users/24ac1e73cb934f11a3ef72b19f38a83c/roles/b43d5a8247e244869c2c70f986f42e8a => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 10 13:10:18.518015 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d949366-e02a-4477-8c28-f491df1f1452 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:10:18.528202 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d949366-e02a-4477-8c28-f491df1f1452 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:10:18.586901 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d24303d-b61c-42ee-b285-0a00846b7a7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:10:18.587565 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d24303d-b61c-42ee-b285-0a00846b7a7c admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:18.587848 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d24303d-b61c-42ee-b285-0a00846b7a7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:18.588289 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d24303d-b61c-42ee-b285-0a00846b7a7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:18.589129 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d24303d-b61c-42ee-b285-0a00846b7a7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:10:18.590733 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d24303d-b61c-42ee-b285-0a00846b7a7c admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:10:18.601583 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1201/2422] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:10:18 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:10:18.605793 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7f5cbc-f4f5-481b-b13b-afc10f927aa5 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:10:18.619499 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7f5cbc-f4f5-481b-b13b-afc10f927aa5 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:10:18.624241 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27064338-1b30-456b-8af8-6d445b23cdce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:10:18.625122 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27064338-1b30-456b-8af8-6d445b23cdce admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:18.625400 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27064338-1b30-456b-8af8-6d445b23cdce admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:18.625673 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27064338-1b30-456b-8af8-6d445b23cdce admin admin] PATH_INFO: `/v3/projects/938a97d6ab4d499882c09b9ef25cc47a/users/24ac1e73cb934f11a3ef72b19f38a83c/roles/b43d5a8247e244869c2c70f986f42e8a` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:18.627602 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27064338-1b30-456b-8af8-6d445b23cdce admin admin] RBAC: Authorizing `identity:create_grant(project_id=938a97d6ab4d499882c09b9ef25cc47a, user_id=24ac1e73cb934f11a3ef72b19f38a83c, role_id=b43d5a8247e244869c2c70f986f42e8a)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:10:18.630230 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27064338-1b30-456b-8af8-6d445b23cdce admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:10:18.640188 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1222/2423] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:10:18 2022] PUT /identity/v3/projects/938a97d6ab4d499882c09b9ef25cc47a/users/24ac1e73cb934f11a3ef72b19f38a83c/roles/b43d5a8247e244869c2c70f986f42e8a => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:10:18.650128 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c1525d-f35d-49b8-94c8-2a6ee5396a7f None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:18.650424 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c1525d-f35d-49b8-94c8-2a6ee5396a7f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:18.650705 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c1525d-f35d-49b8-94c8-2a6ee5396a7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:18.677530 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-d2c1525d-f35d-49b8-94c8-2a6ee5396a7f None None] MFA Rules not processed for user `24ac1e73cb934f11a3ef72b19f38a83c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:10:18.678756 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d2c1525d-f35d-49b8-94c8-2a6ee5396a7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:10:18.697480 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:10:18.747540 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1202/2424] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:10:18 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:10:18.758727 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27064338-1b30-456b-8af8-6d445b23cdce admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:10:18.760436 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-27064338-1b30-456b-8af8-6d445b23cdce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:10:18.817500 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27064338-1b30-456b-8af8-6d445b23cdce admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:10:18.821089 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae057e53-78ea-4fd2-b55b-0d644a0ca671 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6cc95a563b941ac92ab283aa0d5a5d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:10:18.821729 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae057e53-78ea-4fd2-b55b-0d644a0ca671 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:18.822012 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae057e53-78ea-4fd2-b55b-0d644a0ca671 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:18.822288 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae057e53-78ea-4fd2-b55b-0d644a0ca671 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:18.831528 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae057e53-78ea-4fd2-b55b-0d644a0ca671 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:10:18.832972 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae057e53-78ea-4fd2-b55b-0d644a0ca671 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:10:18.842508 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1223/2425] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:10:18 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:10:19.351973 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c1525d-f35d-49b8-94c8-2a6ee5396a7f None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:10:19.418503 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c1525d-f35d-49b8-94c8-2a6ee5396a7f None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:10:19.423070 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd2c7f2-7b42-422f-b5a9-534d84f8dc92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:10:19.424006 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd2c7f2-7b42-422f-b5a9-534d84f8dc92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:19.424286 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd2c7f2-7b42-422f-b5a9-534d84f8dc92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:19.424588 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd2c7f2-7b42-422f-b5a9-534d84f8dc92 admin admin] PATH_INFO: `/v3/users/8cb6680f77bc43eebab12582a707b5a3` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:19.425768 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd2c7f2-7b42-422f-b5a9-534d84f8dc92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8cb6680f77bc43eebab12582a707b5a3)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:10:19.427670 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd2c7f2-7b42-422f-b5a9-534d84f8dc92 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:10:19.590590 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-1fd2c7f2-7b42-422f-b5a9-534d84f8dc92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8cb6680f77bc43eebab12582a707b5a3'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:19.597739 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-1fd2c7f2-7b42-422f-b5a9-534d84f8dc92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8cb6680f77bc43eebab12582a707b5a3'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:19.645260 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-1fd2c7f2-7b42-422f-b5a9-534d84f8dc92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8cb6680f77bc43eebab12582a707b5a3'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:19.645917 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-1fd2c7f2-7b42-422f-b5a9-534d84f8dc92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8cb6680f77bc43eebab12582a707b5a3'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:19.658250 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-1fd2c7f2-7b42-422f-b5a9-534d84f8dc92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8cb6680f77bc43eebab12582a707b5a3'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:19.664567 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1203/2426] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:10:19 2022] DELETE /identity/v3/users/8cb6680f77bc43eebab12582a707b5a3 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:10:19.669160 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc7e108-8b00-4b73-96de-f48385b42689 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:19.669620 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc7e108-8b00-4b73-96de-f48385b42689 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:19.669714 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc7e108-8b00-4b73-96de-f48385b42689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:19.687532 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-efc7e108-8b00-4b73-96de-f48385b42689 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:10:19.688828 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-efc7e108-8b00-4b73-96de-f48385b42689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:10:19.747321 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1224/2427] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:10:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:10:19.757332 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd2c7f2-7b42-422f-b5a9-534d84f8dc92 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:10:19.758745 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-1fd2c7f2-7b42-422f-b5a9-534d84f8dc92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:10:19.798539 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:10:19.815920 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd2c7f2-7b42-422f-b5a9-534d84f8dc92 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:10:19.819538 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ab685f-3fee-42c1-ae61-2daa18835eef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:10:19.820216 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ab685f-3fee-42c1-ae61-2daa18835eef service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:19.820504 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ab685f-3fee-42c1-ae61-2daa18835eef service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:19.820781 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ab685f-3fee-42c1-ae61-2daa18835eef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:19.829910 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ab685f-3fee-42c1-ae61-2daa18835eef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:10:19.831356 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ab685f-3fee-42c1-ae61-2daa18835eef service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:10:19.840818 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1204/2428] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:10:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:10:20.272630 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc7e108-8b00-4b73-96de-f48385b42689 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:10:20.272630 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-efc7e108-8b00-4b73-96de-f48385b42689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:10:20.288858 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc7e108-8b00-4b73-96de-f48385b42689 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:10:20.293594 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a053b0-bba0-4c36-9957-fea9082c8884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:10:20.294466 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a053b0-bba0-4c36-9957-fea9082c8884 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:20.294756 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a053b0-bba0-4c36-9957-fea9082c8884 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:20.295046 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a053b0-bba0-4c36-9957-fea9082c8884 admin admin] PATH_INFO: `/v3/projects/77af84c4bc7745c2ba3e8af2c63e5229` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:20.296375 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a053b0-bba0-4c36-9957-fea9082c8884 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77af84c4bc7745c2ba3e8af2c63e5229)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:10:20.298133 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a053b0-bba0-4c36-9957-fea9082c8884 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:10:20.424500 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:10:20.441187 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b0a053b0-bba0-4c36-9957-fea9082c8884 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77af84c4bc7745c2ba3e8af2c63e5229', 'request_id': 'req-b0a053b0-bba0-4c36-9957-fea9082c8884'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:20.446577 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b0a053b0-bba0-4c36-9957-fea9082c8884 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77af84c4bc7745c2ba3e8af2c63e5229', 'request_id': 'req-b0a053b0-bba0-4c36-9957-fea9082c8884'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:20.452929 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b0a053b0-bba0-4c36-9957-fea9082c8884 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77af84c4bc7745c2ba3e8af2c63e5229', 'request_id': 'req-b0a053b0-bba0-4c36-9957-fea9082c8884'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:20.531475 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b0a053b0-bba0-4c36-9957-fea9082c8884 admin admin] The token cache is being invalidate because project 77af84c4bc7745c2ba3e8af2c63e5229 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:10:20.531770 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b0a053b0-bba0-4c36-9957-fea9082c8884 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:20.532510 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-b0a053b0-bba0-4c36-9957-fea9082c8884 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:20.537631 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1225/2429] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:10:20 2022] DELETE /identity/v3/projects/77af84c4bc7745c2ba3e8af2c63e5229 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:10:20.538822 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ab685f-3fee-42c1-ae61-2daa18835eef service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:10:20.549190 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ab685f-3fee-42c1-ae61-2daa18835eef service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:10:20.612779 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c7e646-ce8c-4424-b08d-89ed4a2de56d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:10:20.613675 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c7e646-ce8c-4424-b08d-89ed4a2de56d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:20.613959 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c7e646-ce8c-4424-b08d-89ed4a2de56d admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:20.614235 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c7e646-ce8c-4424-b08d-89ed4a2de56d admin admin] PATH_INFO: `/v3/users/24ac1e73cb934f11a3ef72b19f38a83c` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:20.615479 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c7e646-ce8c-4424-b08d-89ed4a2de56d admin admin] RBAC: Authorizing `identity:delete_user(user_id=24ac1e73cb934f11a3ef72b19f38a83c)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:10:20.617420 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c7e646-ce8c-4424-b08d-89ed4a2de56d admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:10:20.847521 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-12c7e646-ce8c-4424-b08d-89ed4a2de56d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24ac1e73cb934f11a3ef72b19f38a83c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:20.857039 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-12c7e646-ce8c-4424-b08d-89ed4a2de56d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24ac1e73cb934f11a3ef72b19f38a83c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:20.909152 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-12c7e646-ce8c-4424-b08d-89ed4a2de56d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24ac1e73cb934f11a3ef72b19f38a83c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:20.909907 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-12c7e646-ce8c-4424-b08d-89ed4a2de56d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24ac1e73cb934f11a3ef72b19f38a83c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:20.926026 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-12c7e646-ce8c-4424-b08d-89ed4a2de56d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24ac1e73cb934f11a3ef72b19f38a83c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:20.933116 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1205/2430] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:10:20 2022] DELETE /identity/v3/users/24ac1e73cb934f11a3ef72b19f38a83c => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:10:21.409828 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a053b0-bba0-4c36-9957-fea9082c8884 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:10:21.411359 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b0a053b0-bba0-4c36-9957-fea9082c8884 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:10:21.426322 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a053b0-bba0-4c36-9957-fea9082c8884 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:10:21.500016 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d9b8f1-101a-41b0-9dc4-e285c65903bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:10:21.500880 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d9b8f1-101a-41b0-9dc4-e285c65903bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:21.501158 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d9b8f1-101a-41b0-9dc4-e285c65903bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:21.501427 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d9b8f1-101a-41b0-9dc4-e285c65903bf admin admin] PATH_INFO: `/v3/projects/938a97d6ab4d499882c09b9ef25cc47a` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:21.502627 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d9b8f1-101a-41b0-9dc4-e285c65903bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=938a97d6ab4d499882c09b9ef25cc47a)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:10:21.504630 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d9b8f1-101a-41b0-9dc4-e285c65903bf admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:10:21.605969 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-76d9b8f1-101a-41b0-9dc4-e285c65903bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '938a97d6ab4d499882c09b9ef25cc47a', 'request_id': 'req-76d9b8f1-101a-41b0-9dc4-e285c65903bf'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:21.612464 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-76d9b8f1-101a-41b0-9dc4-e285c65903bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '938a97d6ab4d499882c09b9ef25cc47a', 'request_id': 'req-76d9b8f1-101a-41b0-9dc4-e285c65903bf'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:21.618795 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-76d9b8f1-101a-41b0-9dc4-e285c65903bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '938a97d6ab4d499882c09b9ef25cc47a', 'request_id': 'req-76d9b8f1-101a-41b0-9dc4-e285c65903bf'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:21.690737 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-76d9b8f1-101a-41b0-9dc4-e285c65903bf admin admin] The token cache is being invalidate because project 938a97d6ab4d499882c09b9ef25cc47a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:10:21.691029 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-76d9b8f1-101a-41b0-9dc4-e285c65903bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:21.691674 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-76d9b8f1-101a-41b0-9dc4-e285c65903bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:21.693448 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1226/2431] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:10:21 2022] DELETE /identity/v3/projects/938a97d6ab4d499882c09b9ef25cc47a => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:10:21.703074 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1274aa-32f3-4fea-a796-e0327ee99be5 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:21.703074 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1274aa-32f3-4fea-a796-e0327ee99be5 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:21.703535 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1274aa-32f3-4fea-a796-e0327ee99be5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:21.733460 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-2c1274aa-32f3-4fea-a796-e0327ee99be5 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:10:21.735443 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-2c1274aa-32f3-4fea-a796-e0327ee99be5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:10:21.794972 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:10:21.805691 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1206/2432] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:10:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:10:21.816530 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538303f9-38aa-496d-86ba-ba6c0e3b21ba None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:21.816973 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538303f9-38aa-496d-86ba-ba6c0e3b21ba None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:21.817383 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538303f9-38aa-496d-86ba-ba6c0e3b21ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:21.835806 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-538303f9-38aa-496d-86ba-ba6c0e3b21ba None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:10:21.837174 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-538303f9-38aa-496d-86ba-ba6c0e3b21ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:10:21.845328 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1227/2433] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:10:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:10:21.850256 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1274aa-32f3-4fea-a796-e0327ee99be5 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:10:21.860561 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1274aa-32f3-4fea-a796-e0327ee99be5 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:10:21.865773 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fcc7b7-32aa-4b6f-845d-c5935cf261d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:10:21.866581 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fcc7b7-32aa-4b6f-845d-c5935cf261d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:21.867052 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fcc7b7-32aa-4b6f-845d-c5935cf261d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:21.867333 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fcc7b7-32aa-4b6f-845d-c5935cf261d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:21.868100 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fcc7b7-32aa-4b6f-845d-c5935cf261d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 13:10:21.868417 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fcc7b7-32aa-4b6f-845d-c5935cf261d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:10:21.870015 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fcc7b7-32aa-4b6f-845d-c5935cf261d4 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:10:21.882400 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1207/2434] 10.222.0.27 () {58 vars in 1152 bytes} [Fri Jun 10 13:10:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 13:10:21.890313 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538303f9-38aa-496d-86ba-ba6c0e3b21ba None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:10:21.896364 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538303f9-38aa-496d-86ba-ba6c0e3b21ba None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:10:21.900985 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab310b25-eeba-4fb5-934d-e806cc4ae193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:10:21.901627 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab310b25-eeba-4fb5-934d-e806cc4ae193 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:21.901908 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab310b25-eeba-4fb5-934d-e806cc4ae193 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:21.902182 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab310b25-eeba-4fb5-934d-e806cc4ae193 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:21.903113 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab310b25-eeba-4fb5-934d-e806cc4ae193 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:10:21.905013 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab310b25-eeba-4fb5-934d-e806cc4ae193 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:10:21.995527 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1228/2435] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:10:21 2022] POST /identity/v3/projects => generated 383 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:10:22.000015 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fcc7b7-32aa-4b6f-845d-c5935cf261d4 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:10:22.015148 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fcc7b7-32aa-4b6f-845d-c5935cf261d4 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:10:22.090430 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0734c6-90c3-46de-bd19-03e9bab8cda8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:10:22.091150 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0734c6-90c3-46de-bd19-03e9bab8cda8 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:22.091431 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0734c6-90c3-46de-bd19-03e9bab8cda8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:22.091711 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0734c6-90c3-46de-bd19-03e9bab8cda8 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:22.092435 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0734c6-90c3-46de-bd19-03e9bab8cda8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:10:22.094345 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0734c6-90c3-46de-bd19-03e9bab8cda8 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:10:22.194075 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1208/2436] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:10:21 2022] POST /identity/v3/users => generated 292 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:10:22.195339 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab310b25-eeba-4fb5-934d-e806cc4ae193 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:10:22.207584 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab310b25-eeba-4fb5-934d-e806cc4ae193 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:10:22.213989 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b28084-82a4-424f-85f8-684215892c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:10:22.214676 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b28084-82a4-424f-85f8-684215892c99 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:22.214964 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b28084-82a4-424f-85f8-684215892c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:22.215245 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b28084-82a4-424f-85f8-684215892c99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:22.216075 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b28084-82a4-424f-85f8-684215892c99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:10:22.217676 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b28084-82a4-424f-85f8-684215892c99 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:10:22.229321 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1229/2437] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:10:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:10:22.233508 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0734c6-90c3-46de-bd19-03e9bab8cda8 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:10:22.244136 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0734c6-90c3-46de-bd19-03e9bab8cda8 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:10:22.248838 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c57d8b-66ff-427b-b45b-985bd31fb718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:10:22.249515 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c57d8b-66ff-427b-b45b-985bd31fb718 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:22.249805 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c57d8b-66ff-427b-b45b-985bd31fb718 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:22.250090 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c57d8b-66ff-427b-b45b-985bd31fb718 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:22.250888 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c57d8b-66ff-427b-b45b-985bd31fb718 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:10:22.253050 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c57d8b-66ff-427b-b45b-985bd31fb718 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:10:22.264151 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1209/2438] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:10:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:10:22.269836 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b28084-82a4-424f-85f8-684215892c99 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:10:22.276381 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:10:22.280869 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b28084-82a4-424f-85f8-684215892c99 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:10:22.285545 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735df024-0d37-48e9-a0e3-e630ded74797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:10:22.286414 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735df024-0d37-48e9-a0e3-e630ded74797 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:22.286697 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735df024-0d37-48e9-a0e3-e630ded74797 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:22.286973 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735df024-0d37-48e9-a0e3-e630ded74797 admin admin] PATH_INFO: `/v3/projects/e00cab48ee3444af87c8727fb6c0838e/users/b41cab841fe74793a4dc7ffd7aa2fcdc/roles/43106c664e1b40de984c0c62701149c8` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:22.301975 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735df024-0d37-48e9-a0e3-e630ded74797 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e00cab48ee3444af87c8727fb6c0838e, user_id=b41cab841fe74793a4dc7ffd7aa2fcdc, role_id=43106c664e1b40de984c0c62701149c8)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:10:22.304745 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735df024-0d37-48e9-a0e3-e630ded74797 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:10:22.435697 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1230/2439] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:10:22 2022] PUT /identity/v3/projects/e00cab48ee3444af87c8727fb6c0838e/users/b41cab841fe74793a4dc7ffd7aa2fcdc/roles/43106c664e1b40de984c0c62701149c8 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:10:22.447239 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df084665-84bb-43e3-aef2-29d5544dfe21 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:22.447523 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df084665-84bb-43e3-aef2-29d5544dfe21 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:22.448065 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df084665-84bb-43e3-aef2-29d5544dfe21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:22.476080 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-df084665-84bb-43e3-aef2-29d5544dfe21 None None] MFA Rules not processed for user `b41cab841fe74793a4dc7ffd7aa2fcdc`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:10:22.477456 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-df084665-84bb-43e3-aef2-29d5544dfe21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:10:22.545863 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1210/2440] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:10:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:10:22.550299 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735df024-0d37-48e9-a0e3-e630ded74797 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:10:22.612724 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735df024-0d37-48e9-a0e3-e630ded74797 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:10:22.617373 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dcbc0e-60f3-4467-94ed-910e2a61ea19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:10:22.618036 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dcbc0e-60f3-4467-94ed-910e2a61ea19 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:22.618321 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dcbc0e-60f3-4467-94ed-910e2a61ea19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:22.618599 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dcbc0e-60f3-4467-94ed-910e2a61ea19 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:22.619294 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dcbc0e-60f3-4467-94ed-910e2a61ea19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:10:22.621205 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dcbc0e-60f3-4467-94ed-910e2a61ea19 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:10:22.706599 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1231/2441] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:10:22 2022] POST /identity/v3/projects => generated 383 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:10:22.708182 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df084665-84bb-43e3-aef2-29d5544dfe21 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:10:22.721049 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df084665-84bb-43e3-aef2-29d5544dfe21 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:10:22.780684 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:10:22.790044 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fb02e0-69ef-44ea-ad90-092cc03b5588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:10:22.792718 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fb02e0-69ef-44ea-ad90-092cc03b5588 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:22.792718 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fb02e0-69ef-44ea-ad90-092cc03b5588 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:22.792718 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fb02e0-69ef-44ea-ad90-092cc03b5588 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:22.792718 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fb02e0-69ef-44ea-ad90-092cc03b5588 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:10:22.794300 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fb02e0-69ef-44ea-ad90-092cc03b5588 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:10:22.909754 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1211/2442] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:10:22 2022] POST /identity/v3/users => generated 298 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:10:22.913710 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dcbc0e-60f3-4467-94ed-910e2a61ea19 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:10:22.925578 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dcbc0e-60f3-4467-94ed-910e2a61ea19 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:10:22.940968 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8231986-60f1-4428-8707-28ec32aeec2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:10:22.941661 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8231986-60f1-4428-8707-28ec32aeec2a admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:22.941966 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8231986-60f1-4428-8707-28ec32aeec2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:22.942247 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8231986-60f1-4428-8707-28ec32aeec2a admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:22.943846 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8231986-60f1-4428-8707-28ec32aeec2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:10:22.944629 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8231986-60f1-4428-8707-28ec32aeec2a admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:10:22.961170 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1232/2443] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:10:22 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:10:22.965307 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fb02e0-69ef-44ea-ad90-092cc03b5588 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:10:22.977374 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fb02e0-69ef-44ea-ad90-092cc03b5588 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:10:22.983103 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a16ff5-7a3f-41f8-93ad-c27f98c98dcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:10:22.984163 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a16ff5-7a3f-41f8-93ad-c27f98c98dcf admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:22.984548 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a16ff5-7a3f-41f8-93ad-c27f98c98dcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:22.984976 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a16ff5-7a3f-41f8-93ad-c27f98c98dcf admin admin] PATH_INFO: `/v3/projects/0d158785e5ae43b89ca5b13d9a9553d9/users/30cec0ddf9d94cfb8b5d3e07ac07c682/roles/b43d5a8247e244869c2c70f986f42e8a` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:23.007445 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a16ff5-7a3f-41f8-93ad-c27f98c98dcf admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d158785e5ae43b89ca5b13d9a9553d9, user_id=30cec0ddf9d94cfb8b5d3e07ac07c682, role_id=b43d5a8247e244869c2c70f986f42e8a)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:10:23.010475 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a16ff5-7a3f-41f8-93ad-c27f98c98dcf admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:10:23.012398 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d7cf6c-8da1-4011-94b5-6cb2a6ebacf4 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:23.012694 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d7cf6c-8da1-4011-94b5-6cb2a6ebacf4 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:23.012978 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d7cf6c-8da1-4011-94b5-6cb2a6ebacf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:23.031236 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-33d7cf6c-8da1-4011-94b5-6cb2a6ebacf4 None None] MFA Rules not processed for user `da66362070694e91bbe2cfbbd161b8a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:10:23.032505 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-33d7cf6c-8da1-4011-94b5-6cb2a6ebacf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:10:23.082616 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1212/2444] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:10:22 2022] PUT /identity/v3/projects/0d158785e5ae43b89ca5b13d9a9553d9/users/30cec0ddf9d94cfb8b5d3e07ac07c682/roles/b43d5a8247e244869c2c70f986f42e8a => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 10 13:10:23.086211 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a16ff5-7a3f-41f8-93ad-c27f98c98dcf admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:10:23.114081 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1233/2445] 10.222.0.27 () {58 vars in 967 bytes} [Fri Jun 10 13:10:23 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 10 13:10:23.173767 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a16ff5-7a3f-41f8-93ad-c27f98c98dcf admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:10:23.180330 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0678f48-bb91-40c0-adfd-79b607d27930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:10:23.180617 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0678f48-bb91-40c0-adfd-79b607d27930 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:23.181235 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0678f48-bb91-40c0-adfd-79b607d27930 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:23.181640 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0678f48-bb91-40c0-adfd-79b607d27930 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:23.188544 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0678f48-bb91-40c0-adfd-79b607d27930 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:10:23.188544 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0678f48-bb91-40c0-adfd-79b607d27930 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:10:23.196165 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1213/2446] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:10:23 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 10 13:10:23.200258 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d7cf6c-8da1-4011-94b5-6cb2a6ebacf4 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:10:23.212249 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d7cf6c-8da1-4011-94b5-6cb2a6ebacf4 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:10:23.217050 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ed9d17-200d-4e7a-b64f-dbe1d9ff60c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:10:23.217960 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ed9d17-200d-4e7a-b64f-dbe1d9ff60c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:23.218231 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ed9d17-200d-4e7a-b64f-dbe1d9ff60c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:23.218511 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ed9d17-200d-4e7a-b64f-dbe1d9ff60c5 admin admin] PATH_INFO: `/v3/projects/0d158785e5ae43b89ca5b13d9a9553d9/users/30cec0ddf9d94cfb8b5d3e07ac07c682/roles/b43d5a8247e244869c2c70f986f42e8a` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:23.220532 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ed9d17-200d-4e7a-b64f-dbe1d9ff60c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d158785e5ae43b89ca5b13d9a9553d9, user_id=30cec0ddf9d94cfb8b5d3e07ac07c682, role_id=b43d5a8247e244869c2c70f986f42e8a)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:10:23.223726 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ed9d17-200d-4e7a-b64f-dbe1d9ff60c5 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:10:23.234647 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1234/2447] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:10:23 2022] PUT /identity/v3/projects/0d158785e5ae43b89ca5b13d9a9553d9/users/30cec0ddf9d94cfb8b5d3e07ac07c682/roles/b43d5a8247e244869c2c70f986f42e8a => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:10:23.245761 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6118f3-f3c1-4f58-ad6a-eb1cccc70d52 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:23.246067 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6118f3-f3c1-4f58-ad6a-eb1cccc70d52 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:23.246342 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6118f3-f3c1-4f58-ad6a-eb1cccc70d52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:23.277780 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-cd6118f3-f3c1-4f58-ad6a-eb1cccc70d52 None None] MFA Rules not processed for user `30cec0ddf9d94cfb8b5d3e07ac07c682`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:10:23.281839 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-cd6118f3-f3c1-4f58-ad6a-eb1cccc70d52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:10:23.309657 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ed9d17-200d-4e7a-b64f-dbe1d9ff60c5 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:10:23.311451 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-16ed9d17-200d-4e7a-b64f-dbe1d9ff60c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:10:23.400817 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:10:23.438340 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ed9d17-200d-4e7a-b64f-dbe1d9ff60c5 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:10:23.444688 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1214/2448] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:10:23 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:10:23.450741 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb74c8d0-00a6-4353-ba85-4221c7045bb5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb63b9cd74bf438aa25b72cef8337fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:10:23.451606 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb74c8d0-00a6-4353-ba85-4221c7045bb5 service swift] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:23.452051 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb74c8d0-00a6-4353-ba85-4221c7045bb5 service swift] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:23.452396 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb74c8d0-00a6-4353-ba85-4221c7045bb5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:23.459923 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6118f3-f3c1-4f58-ad6a-eb1cccc70d52 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:10:23.461728 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-cd6118f3-f3c1-4f58-ad6a-eb1cccc70d52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:10:23.529426 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:10:23.543332 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb74c8d0-00a6-4353-ba85-4221c7045bb5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:10:23.544265 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6118f3-f3c1-4f58-ad6a-eb1cccc70d52 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:10:23.545271 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb74c8d0-00a6-4353-ba85-4221c7045bb5 service swift] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:10:23.547821 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b117d15c-88d3-4267-8756-527bf9c1adb3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6cc95a563b941ac92ab283aa0d5a5d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:10:23.548495 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b117d15c-88d3-4267-8756-527bf9c1adb3 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:23.548802 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b117d15c-88d3-4267-8756-527bf9c1adb3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:23.549082 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b117d15c-88d3-4267-8756-527bf9c1adb3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:23.555668 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1235/2449] 10.222.0.27 () {60 vars in 1384 bytes} [Fri Jun 10 13:10:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 10 13:10:23.620097 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b117d15c-88d3-4267-8756-527bf9c1adb3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:10:23.621732 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b117d15c-88d3-4267-8756-527bf9c1adb3 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:10:23.632631 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1215/2450] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:10:23 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:10:29.677770 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb74c8d0-00a6-4353-ba85-4221c7045bb5 service swift] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:10:29.687490 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb74c8d0-00a6-4353-ba85-4221c7045bb5 service swift] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:10:29.691187 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c76cb2-9f07-4fcd-ad8c-e5fa4cc454de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6cc95a563b941ac92ab283aa0d5a5d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:10:29.697880 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c76cb2-9f07-4fcd-ad8c-e5fa4cc454de service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:29.697880 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c76cb2-9f07-4fcd-ad8c-e5fa4cc454de service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:29.697880 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c76cb2-9f07-4fcd-ad8c-e5fa4cc454de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:29.703180 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c76cb2-9f07-4fcd-ad8c-e5fa4cc454de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:10:29.704772 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c76cb2-9f07-4fcd-ad8c-e5fa4cc454de service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:10:29.716869 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1236/2451] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:10:29 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:10:32.206855 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b117d15c-88d3-4267-8756-527bf9c1adb3 service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:10:32.209006 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b117d15c-88d3-4267-8756-527bf9c1adb3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:10:32.277984 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b117d15c-88d3-4267-8756-527bf9c1adb3 service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:10:32.283086 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10207285-652d-4b85-8293-1a79327a65c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:10:32.284644 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10207285-652d-4b85-8293-1a79327a65c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:32.284924 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10207285-652d-4b85-8293-1a79327a65c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:32.285236 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10207285-652d-4b85-8293-1a79327a65c6 admin admin] PATH_INFO: `/v3/users/5195c19d6ce44b14b78565211d3ac234` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:32.286580 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10207285-652d-4b85-8293-1a79327a65c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5195c19d6ce44b14b78565211d3ac234)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:10:32.288564 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10207285-652d-4b85-8293-1a79327a65c6 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:10:32.447824 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-10207285-652d-4b85-8293-1a79327a65c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5195c19d6ce44b14b78565211d3ac234'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:32.455956 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-10207285-652d-4b85-8293-1a79327a65c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5195c19d6ce44b14b78565211d3ac234'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:32.496616 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-10207285-652d-4b85-8293-1a79327a65c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5195c19d6ce44b14b78565211d3ac234'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:32.497323 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-10207285-652d-4b85-8293-1a79327a65c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5195c19d6ce44b14b78565211d3ac234'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:32.509894 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-10207285-652d-4b85-8293-1a79327a65c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5195c19d6ce44b14b78565211d3ac234'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:32.516082 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1216/2452] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:10:32 2022] DELETE /identity/v3/users/5195c19d6ce44b14b78565211d3ac234 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 10 13:10:32.833866 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c76cb2-9f07-4fcd-ad8c-e5fa4cc454de service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:10:32.844479 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c76cb2-9f07-4fcd-ad8c-e5fa4cc454de service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:10:32.910915 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7642f2-cf58-4e7f-beaa-40177586036b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:10:32.911903 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7642f2-cf58-4e7f-beaa-40177586036b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:32.912359 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7642f2-cf58-4e7f-beaa-40177586036b admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:32.912820 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7642f2-cf58-4e7f-beaa-40177586036b admin admin] PATH_INFO: `/v3/projects/7566f9559d464dcba426b5b404d4e29d` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:32.914524 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7642f2-cf58-4e7f-beaa-40177586036b admin admin] RBAC: Authorizing `identity:delete_project(project_id=7566f9559d464dcba426b5b404d4e29d)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:10:32.916619 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7642f2-cf58-4e7f-beaa-40177586036b admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:10:33.074867 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8b7642f2-cf58-4e7f-beaa-40177586036b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7566f9559d464dcba426b5b404d4e29d', 'request_id': 'req-8b7642f2-cf58-4e7f-beaa-40177586036b'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:33.081494 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8b7642f2-cf58-4e7f-beaa-40177586036b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7566f9559d464dcba426b5b404d4e29d', 'request_id': 'req-8b7642f2-cf58-4e7f-beaa-40177586036b'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:33.088701 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8b7642f2-cf58-4e7f-beaa-40177586036b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7566f9559d464dcba426b5b404d4e29d', 'request_id': 'req-8b7642f2-cf58-4e7f-beaa-40177586036b'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:33.148793 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8b7642f2-cf58-4e7f-beaa-40177586036b admin admin] The token cache is being invalidate because project 7566f9559d464dcba426b5b404d4e29d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:10:33.151655 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8b7642f2-cf58-4e7f-beaa-40177586036b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:33.151655 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8b7642f2-cf58-4e7f-beaa-40177586036b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:33.151655 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1237/2453] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:10:32 2022] DELETE /identity/v3/projects/7566f9559d464dcba426b5b404d4e29d => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:10:33.155669 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10207285-652d-4b85-8293-1a79327a65c6 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:10:33.157168 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-10207285-652d-4b85-8293-1a79327a65c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:10:33.170585 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10207285-652d-4b85-8293-1a79327a65c6 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:10:33.180778 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:10:33.227709 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51ca839-0ea6-4290-988f-39f866bc9ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:10:33.228930 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51ca839-0ea6-4290-988f-39f866bc9ade admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:33.229220 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51ca839-0ea6-4290-988f-39f866bc9ade admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:33.229500 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51ca839-0ea6-4290-988f-39f866bc9ade admin admin] PATH_INFO: `/v3/users/0321863f96a142ad8585602ddfa86946` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:33.230716 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51ca839-0ea6-4290-988f-39f866bc9ade admin admin] RBAC: Authorizing `identity:delete_user(user_id=0321863f96a142ad8585602ddfa86946)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:10:33.232663 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51ca839-0ea6-4290-988f-39f866bc9ade admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:10:33.424652 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-d51ca839-0ea6-4290-988f-39f866bc9ade admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0321863f96a142ad8585602ddfa86946'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:33.431389 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-d51ca839-0ea6-4290-988f-39f866bc9ade admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0321863f96a142ad8585602ddfa86946'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:33.502718 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-d51ca839-0ea6-4290-988f-39f866bc9ade admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0321863f96a142ad8585602ddfa86946'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:33.503371 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-d51ca839-0ea6-4290-988f-39f866bc9ade admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0321863f96a142ad8585602ddfa86946'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:33.517399 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-d51ca839-0ea6-4290-988f-39f866bc9ade admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0321863f96a142ad8585602ddfa86946'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:33.523650 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1217/2454] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:10:33 2022] DELETE /identity/v3/users/0321863f96a142ad8585602ddfa86946 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 10 13:10:33.917427 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7642f2-cf58-4e7f-beaa-40177586036b admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:10:33.918946 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8b7642f2-cf58-4e7f-beaa-40177586036b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:10:33.932884 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:10:33.940129 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7642f2-cf58-4e7f-beaa-40177586036b admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:10:34.043406 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a7b9f6-465c-4993-8961-f8dc6c13193f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:10:34.044328 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a7b9f6-465c-4993-8961-f8dc6c13193f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:34.044611 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a7b9f6-465c-4993-8961-f8dc6c13193f admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:34.045076 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a7b9f6-465c-4993-8961-f8dc6c13193f admin admin] PATH_INFO: `/v3/projects/cbdee25e120247ac9be33bc68d2e82b3` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:34.046293 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a7b9f6-465c-4993-8961-f8dc6c13193f admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbdee25e120247ac9be33bc68d2e82b3)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:10:34.048280 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a7b9f6-465c-4993-8961-f8dc6c13193f admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:10:34.179055 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a0a7b9f6-465c-4993-8961-f8dc6c13193f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbdee25e120247ac9be33bc68d2e82b3', 'request_id': 'req-a0a7b9f6-465c-4993-8961-f8dc6c13193f'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:34.185056 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a0a7b9f6-465c-4993-8961-f8dc6c13193f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbdee25e120247ac9be33bc68d2e82b3', 'request_id': 'req-a0a7b9f6-465c-4993-8961-f8dc6c13193f'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:34.191778 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a0a7b9f6-465c-4993-8961-f8dc6c13193f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbdee25e120247ac9be33bc68d2e82b3', 'request_id': 'req-a0a7b9f6-465c-4993-8961-f8dc6c13193f'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:34.290740 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a0a7b9f6-465c-4993-8961-f8dc6c13193f admin admin] The token cache is being invalidate because project cbdee25e120247ac9be33bc68d2e82b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:10:34.291031 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a0a7b9f6-465c-4993-8961-f8dc6c13193f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:34.291705 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a0a7b9f6-465c-4993-8961-f8dc6c13193f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:34.293289 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1238/2455] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:10:33 2022] DELETE /identity/v3/projects/cbdee25e120247ac9be33bc68d2e82b3 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:10:35.745817 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51ca839-0ea6-4290-988f-39f866bc9ade admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:10:35.747294 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d51ca839-0ea6-4290-988f-39f866bc9ade admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:10:35.760962 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51ca839-0ea6-4290-988f-39f866bc9ade admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:10:35.818296 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df63c552-bc2f-46c5-a3bf-41aab744729d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:10:35.819181 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df63c552-bc2f-46c5-a3bf-41aab744729d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:35.819469 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df63c552-bc2f-46c5-a3bf-41aab744729d admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:35.819754 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df63c552-bc2f-46c5-a3bf-41aab744729d admin admin] PATH_INFO: `/v3/users/b41cab841fe74793a4dc7ffd7aa2fcdc` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:35.820986 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df63c552-bc2f-46c5-a3bf-41aab744729d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b41cab841fe74793a4dc7ffd7aa2fcdc)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:10:35.822906 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df63c552-bc2f-46c5-a3bf-41aab744729d admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:10:36.005685 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:10:36.014989 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-df63c552-bc2f-46c5-a3bf-41aab744729d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b41cab841fe74793a4dc7ffd7aa2fcdc'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:36.022364 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-df63c552-bc2f-46c5-a3bf-41aab744729d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b41cab841fe74793a4dc7ffd7aa2fcdc'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:36.079324 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-df63c552-bc2f-46c5-a3bf-41aab744729d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b41cab841fe74793a4dc7ffd7aa2fcdc'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:36.079911 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-df63c552-bc2f-46c5-a3bf-41aab744729d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b41cab841fe74793a4dc7ffd7aa2fcdc'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:36.096439 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-df63c552-bc2f-46c5-a3bf-41aab744729d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b41cab841fe74793a4dc7ffd7aa2fcdc'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:36.107561 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1218/2456] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:10:35 2022] DELETE /identity/v3/users/b41cab841fe74793a4dc7ffd7aa2fcdc => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:10:36.114375 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fb2641-39cb-4998-8a1f-c3aa44e31e0b None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:36.114870 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fb2641-39cb-4998-8a1f-c3aa44e31e0b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:36.114963 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fb2641-39cb-4998-8a1f-c3aa44e31e0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:36.133506 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-77fb2641-39cb-4998-8a1f-c3aa44e31e0b None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:10:36.134790 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-77fb2641-39cb-4998-8a1f-c3aa44e31e0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:10:36.194113 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1239/2457] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:10:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:10:36.203707 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df63c552-bc2f-46c5-a3bf-41aab744729d admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:10:36.210164 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-df63c552-bc2f-46c5-a3bf-41aab744729d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:10:36.264633 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df63c552-bc2f-46c5-a3bf-41aab744729d admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:10:36.268423 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b894f276-5b3f-4487-82d2-f542e88f2d5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:10:36.269068 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b894f276-5b3f-4487-82d2-f542e88f2d5f service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:36.269354 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b894f276-5b3f-4487-82d2-f542e88f2d5f service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:36.269661 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b894f276-5b3f-4487-82d2-f542e88f2d5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:36.279052 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b894f276-5b3f-4487-82d2-f542e88f2d5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:10:36.280521 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b894f276-5b3f-4487-82d2-f542e88f2d5f service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:10:36.290024 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1219/2458] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:10:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:10:36.704122 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fb2641-39cb-4998-8a1f-c3aa44e31e0b None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:10:36.705551 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-77fb2641-39cb-4998-8a1f-c3aa44e31e0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:10:36.717929 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fb2641-39cb-4998-8a1f-c3aa44e31e0b None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:10:36.723610 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c28c85-0d4b-40e3-b897-bfd3359f7dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:10:36.724557 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c28c85-0d4b-40e3-b897-bfd3359f7dc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:36.724834 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c28c85-0d4b-40e3-b897-bfd3359f7dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:36.725102 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c28c85-0d4b-40e3-b897-bfd3359f7dc3 admin admin] PATH_INFO: `/v3/projects/e00cab48ee3444af87c8727fb6c0838e` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:36.726304 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c28c85-0d4b-40e3-b897-bfd3359f7dc3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e00cab48ee3444af87c8727fb6c0838e)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:10:36.728240 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c28c85-0d4b-40e3-b897-bfd3359f7dc3 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:10:36.844886 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:10:36.859688 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-36c28c85-0d4b-40e3-b897-bfd3359f7dc3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e00cab48ee3444af87c8727fb6c0838e', 'request_id': 'req-36c28c85-0d4b-40e3-b897-bfd3359f7dc3'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:36.867209 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-36c28c85-0d4b-40e3-b897-bfd3359f7dc3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e00cab48ee3444af87c8727fb6c0838e', 'request_id': 'req-36c28c85-0d4b-40e3-b897-bfd3359f7dc3'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:36.873941 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-36c28c85-0d4b-40e3-b897-bfd3359f7dc3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e00cab48ee3444af87c8727fb6c0838e', 'request_id': 'req-36c28c85-0d4b-40e3-b897-bfd3359f7dc3'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:36.947773 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-36c28c85-0d4b-40e3-b897-bfd3359f7dc3 admin admin] The token cache is being invalidate because project e00cab48ee3444af87c8727fb6c0838e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:10:36.948089 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-36c28c85-0d4b-40e3-b897-bfd3359f7dc3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:36.948704 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-36c28c85-0d4b-40e3-b897-bfd3359f7dc3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:36.950271 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1240/2459] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:10:36 2022] DELETE /identity/v3/projects/e00cab48ee3444af87c8727fb6c0838e => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:10:36.954249 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b894f276-5b3f-4487-82d2-f542e88f2d5f service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:10:36.964407 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b894f276-5b3f-4487-82d2-f542e88f2d5f service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:10:37.019725 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adf3b55-9457-4cfd-bcbf-a74e6834e725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:10:37.020609 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adf3b55-9457-4cfd-bcbf-a74e6834e725 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:37.020916 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adf3b55-9457-4cfd-bcbf-a74e6834e725 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:37.022261 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adf3b55-9457-4cfd-bcbf-a74e6834e725 admin admin] PATH_INFO: `/v3/users/30cec0ddf9d94cfb8b5d3e07ac07c682` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:37.023541 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adf3b55-9457-4cfd-bcbf-a74e6834e725 admin admin] RBAC: Authorizing `identity:delete_user(user_id=30cec0ddf9d94cfb8b5d3e07ac07c682)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:10:37.025687 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adf3b55-9457-4cfd-bcbf-a74e6834e725 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:10:37.174069 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-0adf3b55-9457-4cfd-bcbf-a74e6834e725 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30cec0ddf9d94cfb8b5d3e07ac07c682'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:37.181106 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-0adf3b55-9457-4cfd-bcbf-a74e6834e725 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30cec0ddf9d94cfb8b5d3e07ac07c682'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:37.182836 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:10:37.240694 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-0adf3b55-9457-4cfd-bcbf-a74e6834e725 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30cec0ddf9d94cfb8b5d3e07ac07c682'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:37.241670 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-0adf3b55-9457-4cfd-bcbf-a74e6834e725 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30cec0ddf9d94cfb8b5d3e07ac07c682'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:37.254850 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-0adf3b55-9457-4cfd-bcbf-a74e6834e725 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30cec0ddf9d94cfb8b5d3e07ac07c682'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:37.260739 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1220/2460] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:10:36 2022] DELETE /identity/v3/users/30cec0ddf9d94cfb8b5d3e07ac07c682 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:10:37.704164 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c28c85-0d4b-40e3-b897-bfd3359f7dc3 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:10:37.705686 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-36c28c85-0d4b-40e3-b897-bfd3359f7dc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:10:37.730874 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c28c85-0d4b-40e3-b897-bfd3359f7dc3 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:10:37.793773 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613f37c1-8b3d-4b21-9889-84ae5e6f4b74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:10:37.794637 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613f37c1-8b3d-4b21-9889-84ae5e6f4b74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:37.794918 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613f37c1-8b3d-4b21-9889-84ae5e6f4b74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:37.795201 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613f37c1-8b3d-4b21-9889-84ae5e6f4b74 admin admin] PATH_INFO: `/v3/projects/0d158785e5ae43b89ca5b13d9a9553d9` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:37.796411 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613f37c1-8b3d-4b21-9889-84ae5e6f4b74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d158785e5ae43b89ca5b13d9a9553d9)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:10:37.798306 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613f37c1-8b3d-4b21-9889-84ae5e6f4b74 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:10:37.898574 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-613f37c1-8b3d-4b21-9889-84ae5e6f4b74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d158785e5ae43b89ca5b13d9a9553d9', 'request_id': 'req-613f37c1-8b3d-4b21-9889-84ae5e6f4b74'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:37.904493 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-613f37c1-8b3d-4b21-9889-84ae5e6f4b74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d158785e5ae43b89ca5b13d9a9553d9', 'request_id': 'req-613f37c1-8b3d-4b21-9889-84ae5e6f4b74'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:37.910721 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-613f37c1-8b3d-4b21-9889-84ae5e6f4b74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d158785e5ae43b89ca5b13d9a9553d9', 'request_id': 'req-613f37c1-8b3d-4b21-9889-84ae5e6f4b74'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:37.988926 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-613f37c1-8b3d-4b21-9889-84ae5e6f4b74 admin admin] The token cache is being invalidate because project 0d158785e5ae43b89ca5b13d9a9553d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:10:37.989375 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-613f37c1-8b3d-4b21-9889-84ae5e6f4b74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:37.990090 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-613f37c1-8b3d-4b21-9889-84ae5e6f4b74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:37.991834 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1241/2461] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:10:37 2022] DELETE /identity/v3/projects/0d158785e5ae43b89ca5b13d9a9553d9 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 10 13:10:37.997957 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e2b000-c296-4342-87fa-da7f77a790e3 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:37.998421 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e2b000-c296-4342-87fa-da7f77a790e3 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:37.998557 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e2b000-c296-4342-87fa-da7f77a790e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:38.017585 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-34e2b000-c296-4342-87fa-da7f77a790e3 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:10:38.018823 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-34e2b000-c296-4342-87fa-da7f77a790e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:10:38.080420 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1221/2462] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:10:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 10 13:10:38.089791 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1b597b-2658-4fcb-aef4-b9963ecd15f3 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:38.090079 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1b597b-2658-4fcb-aef4-b9963ecd15f3 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:38.090357 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1b597b-2658-4fcb-aef4-b9963ecd15f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:38.109301 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-fa1b597b-2658-4fcb-aef4-b9963ecd15f3 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:10:38.110504 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-fa1b597b-2658-4fcb-aef4-b9963ecd15f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:10:38.119231 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1242/2463] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:10:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:10:38.123629 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e2b000-c296-4342-87fa-da7f77a790e3 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:10:38.133931 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e2b000-c296-4342-87fa-da7f77a790e3 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:10:38.138715 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d08c83d-143c-4129-baea-f67b5262725c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:10:38.139385 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d08c83d-143c-4129-baea-f67b5262725c admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:38.139668 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d08c83d-143c-4129-baea-f67b5262725c admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:38.139988 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d08c83d-143c-4129-baea-f67b5262725c admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:38.140729 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d08c83d-143c-4129-baea-f67b5262725c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 13:10:38.141344 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d08c83d-143c-4129-baea-f67b5262725c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:10:38.143054 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d08c83d-143c-4129-baea-f67b5262725c admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:10:38.160253 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1222/2464] 10.222.0.27 () {58 vars in 1152 bytes} [Fri Jun 10 13:10:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 13:10:38.161428 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1b597b-2658-4fcb-aef4-b9963ecd15f3 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:10:38.171694 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1b597b-2658-4fcb-aef4-b9963ecd15f3 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:10:38.177310 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d391e4-218f-4b28-90fe-f42d06b1bb45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:10:38.177995 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d391e4-218f-4b28-90fe-f42d06b1bb45 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:38.178285 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d391e4-218f-4b28-90fe-f42d06b1bb45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:38.178582 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d391e4-218f-4b28-90fe-f42d06b1bb45 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:38.179276 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d391e4-218f-4b28-90fe-f42d06b1bb45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:10:38.181240 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d391e4-218f-4b28-90fe-f42d06b1bb45 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:10:38.313321 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1243/2465] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:10:38 2022] POST /identity/v3/projects => generated 375 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 10 13:10:38.314423 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d08c83d-143c-4129-baea-f67b5262725c admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:10:38.324784 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d08c83d-143c-4129-baea-f67b5262725c admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:10:38.386499 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088d3f38-1433-4f84-b073-e6de82cbdcfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:10:38.387129 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088d3f38-1433-4f84-b073-e6de82cbdcfa admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:38.387416 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088d3f38-1433-4f84-b073-e6de82cbdcfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:38.387696 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088d3f38-1433-4f84-b073-e6de82cbdcfa admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:38.388438 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088d3f38-1433-4f84-b073-e6de82cbdcfa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:10:38.390353 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088d3f38-1433-4f84-b073-e6de82cbdcfa admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:10:38.488324 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1223/2466] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:10:38 2022] POST /identity/v3/users => generated 288 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 10 13:10:38.489682 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d391e4-218f-4b28-90fe-f42d06b1bb45 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:10:38.501141 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d391e4-218f-4b28-90fe-f42d06b1bb45 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:10:38.506993 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb1afce-cfea-4a4a-bb04-386d83d4d4d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:10:38.507735 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb1afce-cfea-4a4a-bb04-386d83d4d4d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:38.508047 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb1afce-cfea-4a4a-bb04-386d83d4d4d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:38.508322 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb1afce-cfea-4a4a-bb04-386d83d4d4d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:38.509048 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb1afce-cfea-4a4a-bb04-386d83d4d4d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:10:38.510626 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb1afce-cfea-4a4a-bb04-386d83d4d4d5 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:10:38.522120 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1244/2467] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:10:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:10:38.526289 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088d3f38-1433-4f84-b073-e6de82cbdcfa admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:10:38.538845 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088d3f38-1433-4f84-b073-e6de82cbdcfa admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:10:38.543911 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e36e283-7910-4707-9bbe-03da861c3455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:10:38.544649 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e36e283-7910-4707-9bbe-03da861c3455 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:38.545020 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e36e283-7910-4707-9bbe-03da861c3455 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:38.545470 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e36e283-7910-4707-9bbe-03da861c3455 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:38.546195 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e36e283-7910-4707-9bbe-03da861c3455 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:10:38.547743 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e36e283-7910-4707-9bbe-03da861c3455 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:10:38.558275 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1224/2468] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:10:38 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:10:38.562451 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb1afce-cfea-4a4a-bb04-386d83d4d4d5 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:10:38.580021 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb1afce-cfea-4a4a-bb04-386d83d4d4d5 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:10:38.580021 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8366d9b-5541-47e4-92ca-ffd7d790fd28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:10:38.580021 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8366d9b-5541-47e4-92ca-ffd7d790fd28 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:38.580021 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8366d9b-5541-47e4-92ca-ffd7d790fd28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:38.581090 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8366d9b-5541-47e4-92ca-ffd7d790fd28 admin admin] PATH_INFO: `/v3/projects/d43e1e2224b9446da4b69f3289139f6d/users/44b57317722246f4a678f1e7a055c510/roles/43106c664e1b40de984c0c62701149c8` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:38.582135 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:10:38.593654 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8366d9b-5541-47e4-92ca-ffd7d790fd28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d43e1e2224b9446da4b69f3289139f6d, user_id=44b57317722246f4a678f1e7a055c510, role_id=43106c664e1b40de984c0c62701149c8)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:10:38.597278 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8366d9b-5541-47e4-92ca-ffd7d790fd28 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:10:38.669566 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457fef56-6718-4169-a6d2-6db471439446 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:38.669862 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457fef56-6718-4169-a6d2-6db471439446 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:38.670185 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457fef56-6718-4169-a6d2-6db471439446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:38.687611 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-457fef56-6718-4169-a6d2-6db471439446 None None] MFA Rules not processed for user `8e32dbeb85bd4fb889d6dde932c60853`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:10:38.688859 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-457fef56-6718-4169-a6d2-6db471439446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:10:38.691480 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1245/2469] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:10:38 2022] PUT /identity/v3/projects/d43e1e2224b9446da4b69f3289139f6d/users/44b57317722246f4a678f1e7a055c510/roles/43106c664e1b40de984c0c62701149c8 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:10:38.702499 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c560cd8-1b36-4ffa-bebc-ac4e3c5f23de None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:38.702783 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c560cd8-1b36-4ffa-bebc-ac4e3c5f23de None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:38.703063 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c560cd8-1b36-4ffa-bebc-ac4e3c5f23de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:38.727914 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:10:38.731868 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-8c560cd8-1b36-4ffa-bebc-ac4e3c5f23de None None] MFA Rules not processed for user `44b57317722246f4a678f1e7a055c510`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:10:38.733094 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8c560cd8-1b36-4ffa-bebc-ac4e3c5f23de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:10:38.749186 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1225/2470] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:10:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 10 13:10:38.802517 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1246/2471] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:10:38 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 10 13:10:38.815292 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457fef56-6718-4169-a6d2-6db471439446 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:10:38.816726 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-457fef56-6718-4169-a6d2-6db471439446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:10:38.881181 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457fef56-6718-4169-a6d2-6db471439446 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:10:38.884818 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4b2838-cdba-4480-be34-0fd95e7f9619 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6cc95a563b941ac92ab283aa0d5a5d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:10:38.885485 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4b2838-cdba-4480-be34-0fd95e7f9619 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:38.885749 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4b2838-cdba-4480-be34-0fd95e7f9619 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:38.886023 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4b2838-cdba-4480-be34-0fd95e7f9619 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:38.894987 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4b2838-cdba-4480-be34-0fd95e7f9619 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:10:38.896499 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4b2838-cdba-4480-be34-0fd95e7f9619 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:10:38.905887 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1226/2472] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:10:38 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:10:48.799373 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=38,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:10:48.901308 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=33 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:10:55.846275 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c560cd8-1b36-4ffa-bebc-ac4e3c5f23de None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:10:55.906558 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c560cd8-1b36-4ffa-bebc-ac4e3c5f23de None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:10:55.910952 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f703bd35-0b98-4b7f-b05f-a2ed0e99294f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:10:55.912321 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f703bd35-0b98-4b7f-b05f-a2ed0e99294f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:55.912600 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f703bd35-0b98-4b7f-b05f-a2ed0e99294f admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:55.912873 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f703bd35-0b98-4b7f-b05f-a2ed0e99294f admin admin] PATH_INFO: `/v3/users/44b57317722246f4a678f1e7a055c510` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:55.914034 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f703bd35-0b98-4b7f-b05f-a2ed0e99294f admin admin] RBAC: Authorizing `identity:delete_user(user_id=44b57317722246f4a678f1e7a055c510)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:10:55.915961 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f703bd35-0b98-4b7f-b05f-a2ed0e99294f admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:10:56.046645 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f703bd35-0b98-4b7f-b05f-a2ed0e99294f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44b57317722246f4a678f1e7a055c510'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:56.054015 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f703bd35-0b98-4b7f-b05f-a2ed0e99294f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44b57317722246f4a678f1e7a055c510'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:56.096579 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f703bd35-0b98-4b7f-b05f-a2ed0e99294f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44b57317722246f4a678f1e7a055c510'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:56.097186 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f703bd35-0b98-4b7f-b05f-a2ed0e99294f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44b57317722246f4a678f1e7a055c510'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:56.109515 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-f703bd35-0b98-4b7f-b05f-a2ed0e99294f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44b57317722246f4a678f1e7a055c510'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:56.116112 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1247/2473] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:10:55 2022] DELETE /identity/v3/users/44b57317722246f4a678f1e7a055c510 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:10:56.120938 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad27e08-59f1-476b-bea1-a0097e162cb7 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:56.121425 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad27e08-59f1-476b-bea1-a0097e162cb7 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:56.121425 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad27e08-59f1-476b-bea1-a0097e162cb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:56.138779 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-7ad27e08-59f1-476b-bea1-a0097e162cb7 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:10:56.140068 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-7ad27e08-59f1-476b-bea1-a0097e162cb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:10:56.198477 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1227/2474] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:10:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:10:56.210709 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f703bd35-0b98-4b7f-b05f-a2ed0e99294f admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:10:56.212184 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f703bd35-0b98-4b7f-b05f-a2ed0e99294f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:10:56.271203 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f703bd35-0b98-4b7f-b05f-a2ed0e99294f admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:10:56.275034 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc410e2-c5bb-48c7-9157-e80c60702864 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:10:56.275794 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc410e2-c5bb-48c7-9157-e80c60702864 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:56.276217 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc410e2-c5bb-48c7-9157-e80c60702864 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:56.276589 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc410e2-c5bb-48c7-9157-e80c60702864 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:56.287071 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc410e2-c5bb-48c7-9157-e80c60702864 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:10:56.288528 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc410e2-c5bb-48c7-9157-e80c60702864 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:10:56.297860 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1248/2475] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:10:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:10:56.692278 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad27e08-59f1-476b-bea1-a0097e162cb7 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:10:56.692278 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-7ad27e08-59f1-476b-bea1-a0097e162cb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:10:56.702502 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad27e08-59f1-476b-bea1-a0097e162cb7 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:10:56.708360 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bb93e0-8466-4c86-8659-2e9dca3ee6c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:10:56.710227 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bb93e0-8466-4c86-8659-2e9dca3ee6c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:56.710726 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bb93e0-8466-4c86-8659-2e9dca3ee6c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:56.711223 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bb93e0-8466-4c86-8659-2e9dca3ee6c7 admin admin] PATH_INFO: `/v3/projects/d43e1e2224b9446da4b69f3289139f6d` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:56.715148 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bb93e0-8466-4c86-8659-2e9dca3ee6c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d43e1e2224b9446da4b69f3289139f6d)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:10:56.718894 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bb93e0-8466-4c86-8659-2e9dca3ee6c7 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:10:56.840243 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b2bb93e0-8466-4c86-8659-2e9dca3ee6c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd43e1e2224b9446da4b69f3289139f6d', 'request_id': 'req-b2bb93e0-8466-4c86-8659-2e9dca3ee6c7'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:56.845646 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b2bb93e0-8466-4c86-8659-2e9dca3ee6c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd43e1e2224b9446da4b69f3289139f6d', 'request_id': 'req-b2bb93e0-8466-4c86-8659-2e9dca3ee6c7'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:56.851742 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b2bb93e0-8466-4c86-8659-2e9dca3ee6c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd43e1e2224b9446da4b69f3289139f6d', 'request_id': 'req-b2bb93e0-8466-4c86-8659-2e9dca3ee6c7'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:56.913403 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b2bb93e0-8466-4c86-8659-2e9dca3ee6c7 admin admin] The token cache is being invalidate because project d43e1e2224b9446da4b69f3289139f6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:10:56.913685 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b2bb93e0-8466-4c86-8659-2e9dca3ee6c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:56.914291 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b2bb93e0-8466-4c86-8659-2e9dca3ee6c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:10:56.915818 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1228/2476] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:10:56 2022] DELETE /identity/v3/projects/d43e1e2224b9446da4b69f3289139f6d => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:10:56.921201 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48311262-c1b3-4e52-8e60-963cbbc24a8c None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:56.921201 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48311262-c1b3-4e52-8e60-963cbbc24a8c None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:56.921726 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48311262-c1b3-4e52-8e60-963cbbc24a8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:56.938568 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-48311262-c1b3-4e52-8e60-963cbbc24a8c None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:10:56.939694 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-48311262-c1b3-4e52-8e60-963cbbc24a8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:10:56.956565 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:10:56.999243 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1249/2477] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:10:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:10:57.008104 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5ee1ff-aabe-4f4e-91bc-13e11a5c378f None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:57.008390 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5ee1ff-aabe-4f4e-91bc-13e11a5c378f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:57.008664 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5ee1ff-aabe-4f4e-91bc-13e11a5c378f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:57.028914 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-af5ee1ff-aabe-4f4e-91bc-13e11a5c378f None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:10:57.030104 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-af5ee1ff-aabe-4f4e-91bc-13e11a5c378f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:10:57.037900 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1229/2478] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:10:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:10:57.042045 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48311262-c1b3-4e52-8e60-963cbbc24a8c None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:10:57.051492 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48311262-c1b3-4e52-8e60-963cbbc24a8c None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:10:57.056645 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e438c208-05a0-4a87-ae12-77787c3b9bc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:10:57.057296 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e438c208-05a0-4a87-ae12-77787c3b9bc9 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:57.057584 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e438c208-05a0-4a87-ae12-77787c3b9bc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:57.057862 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e438c208-05a0-4a87-ae12-77787c3b9bc9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:57.058593 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e438c208-05a0-4a87-ae12-77787c3b9bc9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 13:10:57.058904 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e438c208-05a0-4a87-ae12-77787c3b9bc9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:10:57.060523 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e438c208-05a0-4a87-ae12-77787c3b9bc9 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:10:57.072332 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1250/2479] 10.222.0.27 () {58 vars in 1152 bytes} [Fri Jun 10 13:10:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 13:10:57.076305 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5ee1ff-aabe-4f4e-91bc-13e11a5c378f None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:10:57.085160 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5ee1ff-aabe-4f4e-91bc-13e11a5c378f None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:10:57.089699 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3e391e-5b0d-4ffd-a2bb-4b2f8d248311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:10:57.090336 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3e391e-5b0d-4ffd-a2bb-4b2f8d248311 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:57.090615 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3e391e-5b0d-4ffd-a2bb-4b2f8d248311 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:57.090890 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3e391e-5b0d-4ffd-a2bb-4b2f8d248311 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:57.091563 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3e391e-5b0d-4ffd-a2bb-4b2f8d248311 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:10:57.093494 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3e391e-5b0d-4ffd-a2bb-4b2f8d248311 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:10:57.196503 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1230/2480] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:10:57 2022] POST /identity/v3/projects => generated 373 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:10:57.200456 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e438c208-05a0-4a87-ae12-77787c3b9bc9 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:10:57.210452 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e438c208-05a0-4a87-ae12-77787c3b9bc9 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:10:57.265711 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7b0492-4fe0-4fff-8aeb-dcaac2ac8958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:10:57.266378 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7b0492-4fe0-4fff-8aeb-dcaac2ac8958 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:57.266695 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7b0492-4fe0-4fff-8aeb-dcaac2ac8958 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:57.266973 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7b0492-4fe0-4fff-8aeb-dcaac2ac8958 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:57.267686 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7b0492-4fe0-4fff-8aeb-dcaac2ac8958 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:10:57.269600 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7b0492-4fe0-4fff-8aeb-dcaac2ac8958 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:10:57.336789 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1251/2481] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:10:57 2022] POST /identity/v3/users => generated 287 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:10:57.340821 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3e391e-5b0d-4ffd-a2bb-4b2f8d248311 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:10:57.351224 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3e391e-5b0d-4ffd-a2bb-4b2f8d248311 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:10:57.355829 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583efce5-1082-4ca8-a911-008ec526b5ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:10:57.356519 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583efce5-1082-4ca8-a911-008ec526b5ac admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:57.356797 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583efce5-1082-4ca8-a911-008ec526b5ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:57.357071 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583efce5-1082-4ca8-a911-008ec526b5ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:57.357793 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583efce5-1082-4ca8-a911-008ec526b5ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:10:57.359415 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583efce5-1082-4ca8-a911-008ec526b5ac admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:10:57.370671 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1231/2482] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:10:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:10:57.377182 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7b0492-4fe0-4fff-8aeb-dcaac2ac8958 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:10:57.387742 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7b0492-4fe0-4fff-8aeb-dcaac2ac8958 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:10:57.392347 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7b7708-7c3f-4353-88b4-40f6c0c7ddeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:10:57.393003 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7b7708-7c3f-4353-88b4-40f6c0c7ddeb admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:57.393287 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7b7708-7c3f-4353-88b4-40f6c0c7ddeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:57.393563 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7b7708-7c3f-4353-88b4-40f6c0c7ddeb admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:57.394286 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7b7708-7c3f-4353-88b4-40f6c0c7ddeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:10:57.395877 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7b7708-7c3f-4353-88b4-40f6c0c7ddeb admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:10:57.406337 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1252/2483] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:10:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:10:57.410417 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583efce5-1082-4ca8-a911-008ec526b5ac admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:10:57.421468 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583efce5-1082-4ca8-a911-008ec526b5ac admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:10:57.426193 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05492ea-0a1d-4983-b7fe-6b154d988fff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:10:57.427102 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05492ea-0a1d-4983-b7fe-6b154d988fff admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:57.427384 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05492ea-0a1d-4983-b7fe-6b154d988fff admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:57.427774 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05492ea-0a1d-4983-b7fe-6b154d988fff admin admin] PATH_INFO: `/v3/projects/0ce61ea5719a40eeb77dc557a849fd82/users/24a6e71e55fc4b4db23689d514b3b879/roles/43106c664e1b40de984c0c62701149c8` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:57.442995 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05492ea-0a1d-4983-b7fe-6b154d988fff admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ce61ea5719a40eeb77dc557a849fd82, user_id=24a6e71e55fc4b4db23689d514b3b879, role_id=43106c664e1b40de984c0c62701149c8)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:10:57.445684 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05492ea-0a1d-4983-b7fe-6b154d988fff admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:10:57.509708 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1232/2484] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:10:57 2022] PUT /identity/v3/projects/0ce61ea5719a40eeb77dc557a849fd82/users/24a6e71e55fc4b4db23689d514b3b879/roles/43106c664e1b40de984c0c62701149c8 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:10:57.514724 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e706707-bbe8-4b23-b4d8-157f2ce7cd36 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:57.515195 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e706707-bbe8-4b23-b4d8-157f2ce7cd36 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:57.515486 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e706707-bbe8-4b23-b4d8-157f2ce7cd36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:57.534120 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-2e706707-bbe8-4b23-b4d8-157f2ce7cd36 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:10:57.535387 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2e706707-bbe8-4b23-b4d8-157f2ce7cd36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:10:57.593161 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1253/2485] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:10:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:10:57.602680 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05492ea-0a1d-4983-b7fe-6b154d988fff admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:10:57.604444 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e05492ea-0a1d-4983-b7fe-6b154d988fff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:10:57.626714 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:10:57.662703 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05492ea-0a1d-4983-b7fe-6b154d988fff admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:10:57.666639 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d36fade-1973-4c5b-a489-769b59e37b07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:10:57.667272 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d36fade-1973-4c5b-a489-769b59e37b07 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:57.667550 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d36fade-1973-4c5b-a489-769b59e37b07 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:57.667819 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d36fade-1973-4c5b-a489-769b59e37b07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:57.676978 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d36fade-1973-4c5b-a489-769b59e37b07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:10:57.678448 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d36fade-1973-4c5b-a489-769b59e37b07 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:10:57.687623 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1233/2486] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:10:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:10:59.158669 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3883043a-ef77-4f39-a0ea-c17982e2cb6a None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:59.159230 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3883043a-ef77-4f39-a0ea-c17982e2cb6a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:59.159230 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3883043a-ef77-4f39-a0ea-c17982e2cb6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:59.186691 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-3883043a-ef77-4f39-a0ea-c17982e2cb6a None None] MFA Rules not processed for user `24a6e71e55fc4b4db23689d514b3b879`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:10:59.188146 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3883043a-ef77-4f39-a0ea-c17982e2cb6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:10:59.208572 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:10:59.257702 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1254/2487] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:10:59 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:10:59.268536 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d36fade-1973-4c5b-a489-769b59e37b07 service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:10:59.270072 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5d36fade-1973-4c5b-a489-769b59e37b07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:10:59.327519 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d36fade-1973-4c5b-a489-769b59e37b07 service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:10:59.331153 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd12eee-f1aa-4d02-80a2-195c637da939 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6cc95a563b941ac92ab283aa0d5a5d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:10:59.331795 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd12eee-f1aa-4d02-80a2-195c637da939 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:10:59.332109 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd12eee-f1aa-4d02-80a2-195c637da939 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:10:59.332389 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd12eee-f1aa-4d02-80a2-195c637da939 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:10:59.340709 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd12eee-f1aa-4d02-80a2-195c637da939 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:10:59.342148 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd12eee-f1aa-4d02-80a2-195c637da939 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:10:59.351016 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1234/2488] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:10:59 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:11:01.017204 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5624868d-efbe-43a1-8d1d-2a2df0fe1547 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:11:01.017687 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5624868d-efbe-43a1-8d1d-2a2df0fe1547 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:11:01.017803 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5624868d-efbe-43a1-8d1d-2a2df0fe1547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:11:01.035240 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-5624868d-efbe-43a1-8d1d-2a2df0fe1547 None None] MFA Rules not processed for user `24a6e71e55fc4b4db23689d514b3b879`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:11:01.036470 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5624868d-efbe-43a1-8d1d-2a2df0fe1547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:11:01.043321 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1255/2489] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:11:01 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:11:09.347073 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=43 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:11:11.030025 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=22 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:11:17.677930 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7432c2-7f15-41ad-b06a-5f75bfaecd25 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:11:17.678407 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7432c2-7f15-41ad-b06a-5f75bfaecd25 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:11:17.678533 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7432c2-7f15-41ad-b06a-5f75bfaecd25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:11:17.695544 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-fa7432c2-7f15-41ad-b06a-5f75bfaecd25 None None] MFA Rules not processed for user `da66362070694e91bbe2cfbbd161b8a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:11:17.696782 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-fa7432c2-7f15-41ad-b06a-5f75bfaecd25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:11:17.756282 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1235/2490] 10.222.0.27 () {58 vars in 1009 bytes} [Fri Jun 10 13:11:17 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:11:17.768309 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5624868d-efbe-43a1-8d1d-2a2df0fe1547 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:11:17.769809 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5624868d-efbe-43a1-8d1d-2a2df0fe1547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:11:17.845761 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5624868d-efbe-43a1-8d1d-2a2df0fe1547 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:11:17.850932 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b00e66-42ce-426d-b29e-1fbb0d0a8a14 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb63b9cd74bf438aa25b72cef8337fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:11:17.852215 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b00e66-42ce-426d-b29e-1fbb0d0a8a14 service swift] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:11:17.852501 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b00e66-42ce-426d-b29e-1fbb0d0a8a14 service swift] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:11:17.852781 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b00e66-42ce-426d-b29e-1fbb0d0a8a14 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:11:17.865151 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b00e66-42ce-426d-b29e-1fbb0d0a8a14 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:11:17.866712 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b00e66-42ce-426d-b29e-1fbb0d0a8a14 service swift] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:11:17.876061 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1256/2491] 10.222.0.27 () {60 vars in 1384 bytes} [Fri Jun 10 13:11:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 10 13:11:19.169495 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19eae57-5280-481a-8ed8-bc522580480c None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:11:19.170065 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19eae57-5280-481a-8ed8-bc522580480c None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:11:19.170065 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19eae57-5280-481a-8ed8-bc522580480c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:11:19.198025 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-a19eae57-5280-481a-8ed8-bc522580480c None None] MFA Rules not processed for user `24a6e71e55fc4b4db23689d514b3b879`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:11:19.199255 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a19eae57-5280-481a-8ed8-bc522580480c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:11:19.207910 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1236/2492] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:11:19 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:11:21.145512 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b00e66-42ce-426d-b29e-1fbb0d0a8a14 service swift] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:11:21.147738 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b3b00e66-42ce-426d-b29e-1fbb0d0a8a14 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:11:21.164357 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b00e66-42ce-426d-b29e-1fbb0d0a8a14 service swift] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:11:21.171077 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9859e93-1f8a-4aa2-a667-92152921eecd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:11:21.171077 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9859e93-1f8a-4aa2-a667-92152921eecd service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:11:21.171077 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9859e93-1f8a-4aa2-a667-92152921eecd service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:11:21.171077 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9859e93-1f8a-4aa2-a667-92152921eecd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:11:21.172668 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d9859e93-1f8a-4aa2-a667-92152921eecd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:11:21.285686 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9859e93-1f8a-4aa2-a667-92152921eecd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:11:21.285686 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9859e93-1f8a-4aa2-a667-92152921eecd service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:11:21.296381 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1257/2493] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:11:21 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:11:26.871736 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19eae57-5280-481a-8ed8-bc522580480c None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:11:26.873530 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a19eae57-5280-481a-8ed8-bc522580480c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:11:26.972354 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19eae57-5280-481a-8ed8-bc522580480c None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:11:26.982190 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ca9f3f-5f36-49fd-9e11-6c09b4ce8a44 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd972ab782b3f4d2aba575314f16dafc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:11:26.983755 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ca9f3f-5f36-49fd-9e11-6c09b4ce8a44 service placement] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:11:26.984229 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ca9f3f-5f36-49fd-9e11-6c09b4ce8a44 service placement] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:11:26.984587 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ca9f3f-5f36-49fd-9e11-6c09b4ce8a44 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:11:26.986664 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-21ca9f3f-5f36-49fd-9e11-6c09b4ce8a44 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:11:27.007088 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ca9f3f-5f36-49fd-9e11-6c09b4ce8a44 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:11:27.008846 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ca9f3f-5f36-49fd-9e11-6c09b4ce8a44 service placement] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:11:27.021611 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1237/2494] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:11:26 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:11:31.291360 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=60 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:11:31.615024 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9859e93-1f8a-4aa2-a667-92152921eecd service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:11:31.623784 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9859e93-1f8a-4aa2-a667-92152921eecd service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:11:31.627540 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2518b2-bebd-408b-9f7e-88731e45535a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:11:31.628291 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2518b2-bebd-408b-9f7e-88731e45535a service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:11:31.628644 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2518b2-bebd-408b-9f7e-88731e45535a service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:11:31.628992 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2518b2-bebd-408b-9f7e-88731e45535a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:11:31.630897 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ef2518b2-bebd-408b-9f7e-88731e45535a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:11:31.643372 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2518b2-bebd-408b-9f7e-88731e45535a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:11:31.644831 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2518b2-bebd-408b-9f7e-88731e45535a service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:11:31.659253 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1258/2495] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:11:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:11:35.580344 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ca9f3f-5f36-49fd-9e11-6c09b4ce8a44 service placement] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:11:35.591904 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ca9f3f-5f36-49fd-9e11-6c09b4ce8a44 service placement] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:11:35.597113 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954c21df-1086-4b77-8393-b07b1105be41 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd972ab782b3f4d2aba575314f16dafc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:11:35.597824 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954c21df-1086-4b77-8393-b07b1105be41 service placement] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:11:35.598110 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954c21df-1086-4b77-8393-b07b1105be41 service placement] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:11:35.598385 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954c21df-1086-4b77-8393-b07b1105be41 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:11:35.600194 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-954c21df-1086-4b77-8393-b07b1105be41 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:11:35.612076 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954c21df-1086-4b77-8393-b07b1105be41 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:11:35.619482 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954c21df-1086-4b77-8393-b07b1105be41 service placement] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:11:35.624591 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1238/2496] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:11:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:11:39.339556 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2518b2-bebd-408b-9f7e-88731e45535a service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:11:39.341187 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ef2518b2-bebd-408b-9f7e-88731e45535a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:11:39.357146 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2518b2-bebd-408b-9f7e-88731e45535a service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:11:39.365962 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610e3fef-6434-483d-afb9-c7b8fc132027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:11:39.367564 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610e3fef-6434-483d-afb9-c7b8fc132027 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:11:39.368077 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610e3fef-6434-483d-afb9-c7b8fc132027 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:11:39.368561 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610e3fef-6434-483d-afb9-c7b8fc132027 admin admin] PATH_INFO: `/v3/users/a72b5e893615479ea3fe1838d07496ba` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:11:39.369900 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610e3fef-6434-483d-afb9-c7b8fc132027 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a72b5e893615479ea3fe1838d07496ba)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:11:39.371952 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610e3fef-6434-483d-afb9-c7b8fc132027 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:11:39.526612 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-610e3fef-6434-483d-afb9-c7b8fc132027 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a72b5e893615479ea3fe1838d07496ba'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:11:39.534594 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-610e3fef-6434-483d-afb9-c7b8fc132027 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a72b5e893615479ea3fe1838d07496ba'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:11:39.599603 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-610e3fef-6434-483d-afb9-c7b8fc132027 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a72b5e893615479ea3fe1838d07496ba'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:11:39.600278 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-610e3fef-6434-483d-afb9-c7b8fc132027 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a72b5e893615479ea3fe1838d07496ba'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:11:39.613222 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-610e3fef-6434-483d-afb9-c7b8fc132027 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a72b5e893615479ea3fe1838d07496ba'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:11:39.619039 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1259/2497] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:11:39 2022] DELETE /identity/v3/users/a72b5e893615479ea3fe1838d07496ba => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:11:39.897618 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954c21df-1086-4b77-8393-b07b1105be41 service placement] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:11:39.908518 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=67 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:11:39.912331 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954c21df-1086-4b77-8393-b07b1105be41 service placement] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:11:39.970933 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a900f1-b099-4bcf-89f4-abe5f9faabd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:11:39.971844 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a900f1-b099-4bcf-89f4-abe5f9faabd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:11:39.972153 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a900f1-b099-4bcf-89f4-abe5f9faabd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:11:39.972427 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a900f1-b099-4bcf-89f4-abe5f9faabd3 admin admin] PATH_INFO: `/v3/projects/b61776123cea47be842ccea6c79dfc12` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:11:39.973581 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a900f1-b099-4bcf-89f4-abe5f9faabd3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b61776123cea47be842ccea6c79dfc12)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:11:39.975496 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a900f1-b099-4bcf-89f4-abe5f9faabd3 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:11:40.074008 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-d8a900f1-b099-4bcf-89f4-abe5f9faabd3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b61776123cea47be842ccea6c79dfc12', 'request_id': 'req-d8a900f1-b099-4bcf-89f4-abe5f9faabd3'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:11:40.079735 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-d8a900f1-b099-4bcf-89f4-abe5f9faabd3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b61776123cea47be842ccea6c79dfc12', 'request_id': 'req-d8a900f1-b099-4bcf-89f4-abe5f9faabd3'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:11:40.086344 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-d8a900f1-b099-4bcf-89f4-abe5f9faabd3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b61776123cea47be842ccea6c79dfc12', 'request_id': 'req-d8a900f1-b099-4bcf-89f4-abe5f9faabd3'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:11:40.158991 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-d8a900f1-b099-4bcf-89f4-abe5f9faabd3 admin admin] The token cache is being invalidate because project b61776123cea47be842ccea6c79dfc12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:11:40.159198 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-d8a900f1-b099-4bcf-89f4-abe5f9faabd3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:11:40.159800 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-d8a900f1-b099-4bcf-89f4-abe5f9faabd3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:11:40.163434 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1239/2498] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:11:39 2022] DELETE /identity/v3/projects/b61776123cea47be842ccea6c79dfc12 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:11:40.165287 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610e3fef-6434-483d-afb9-c7b8fc132027 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:11:40.166653 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-610e3fef-6434-483d-afb9-c7b8fc132027 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:11:40.180196 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610e3fef-6434-483d-afb9-c7b8fc132027 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:11:40.236600 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d962e6eb-bbea-496c-940d-c3d505489789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:11:40.237459 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d962e6eb-bbea-496c-940d-c3d505489789 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:11:40.237738 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d962e6eb-bbea-496c-940d-c3d505489789 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:11:40.238011 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d962e6eb-bbea-496c-940d-c3d505489789 admin admin] PATH_INFO: `/v3/users/869c7d82fde44c4083eeb51dc91b2b61` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:11:40.239170 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d962e6eb-bbea-496c-940d-c3d505489789 admin admin] RBAC: Authorizing `identity:delete_user(user_id=869c7d82fde44c4083eeb51dc91b2b61)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:11:40.241085 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d962e6eb-bbea-496c-940d-c3d505489789 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:11:40.441786 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d962e6eb-bbea-496c-940d-c3d505489789 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '869c7d82fde44c4083eeb51dc91b2b61'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:11:40.449430 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d962e6eb-bbea-496c-940d-c3d505489789 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '869c7d82fde44c4083eeb51dc91b2b61'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:11:40.455139 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:11:40.526660 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d962e6eb-bbea-496c-940d-c3d505489789 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '869c7d82fde44c4083eeb51dc91b2b61'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:11:40.527723 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d962e6eb-bbea-496c-940d-c3d505489789 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '869c7d82fde44c4083eeb51dc91b2b61'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:11:40.550270 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d962e6eb-bbea-496c-940d-c3d505489789 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '869c7d82fde44c4083eeb51dc91b2b61'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:11:40.562590 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1260/2499] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:11:40 2022] DELETE /identity/v3/users/869c7d82fde44c4083eeb51dc91b2b61 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:11:40.807553 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a900f1-b099-4bcf-89f4-abe5f9faabd3 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:11:40.812470 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d8a900f1-b099-4bcf-89f4-abe5f9faabd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:11:40.827829 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a900f1-b099-4bcf-89f4-abe5f9faabd3 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:11:40.912050 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51b7d27-c57d-4c97-be43-38dd3d0a1a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:11:40.912930 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b7d27-c57d-4c97-be43-38dd3d0a1a70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:11:40.913215 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b7d27-c57d-4c97-be43-38dd3d0a1a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:11:40.914189 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b7d27-c57d-4c97-be43-38dd3d0a1a70 admin admin] PATH_INFO: `/v3/projects/9f2d9d35e5e5436a9a3b84313d20fb67` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:11:40.915766 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51b7d27-c57d-4c97-be43-38dd3d0a1a70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f2d9d35e5e5436a9a3b84313d20fb67)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:11:40.917775 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51b7d27-c57d-4c97-be43-38dd3d0a1a70 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:11:41.014096 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:11:41.024821 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b51b7d27-c57d-4c97-be43-38dd3d0a1a70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f2d9d35e5e5436a9a3b84313d20fb67', 'request_id': 'req-b51b7d27-c57d-4c97-be43-38dd3d0a1a70'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:11:41.030676 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b51b7d27-c57d-4c97-be43-38dd3d0a1a70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f2d9d35e5e5436a9a3b84313d20fb67', 'request_id': 'req-b51b7d27-c57d-4c97-be43-38dd3d0a1a70'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:11:41.037043 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b51b7d27-c57d-4c97-be43-38dd3d0a1a70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f2d9d35e5e5436a9a3b84313d20fb67', 'request_id': 'req-b51b7d27-c57d-4c97-be43-38dd3d0a1a70'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:11:41.166735 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b51b7d27-c57d-4c97-be43-38dd3d0a1a70 admin admin] The token cache is being invalidate because project 9f2d9d35e5e5436a9a3b84313d20fb67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:11:41.167021 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b51b7d27-c57d-4c97-be43-38dd3d0a1a70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:11:41.167770 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b51b7d27-c57d-4c97-be43-38dd3d0a1a70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:11:41.181485 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1240/2500] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:11:40 2022] DELETE /identity/v3/projects/9f2d9d35e5e5436a9a3b84313d20fb67 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:11:42.579006 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15416b53-52b7-4c0f-b220-f2566c95f0bc None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:11:42.579467 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15416b53-52b7-4c0f-b220-f2566c95f0bc None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:11:42.584030 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15416b53-52b7-4c0f-b220-f2566c95f0bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:11:42.597098 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-15416b53-52b7-4c0f-b220-f2566c95f0bc None None] MFA Rules not processed for user `da66362070694e91bbe2cfbbd161b8a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:11:42.598312 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-15416b53-52b7-4c0f-b220-f2566c95f0bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:11:42.653438 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1261/2501] 10.222.0.27 () {58 vars in 1009 bytes} [Fri Jun 10 13:11:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:11:42.665329 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51b7d27-c57d-4c97-be43-38dd3d0a1a70 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:11:42.666723 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b51b7d27-c57d-4c97-be43-38dd3d0a1a70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:11:42.736909 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51b7d27-c57d-4c97-be43-38dd3d0a1a70 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:11:42.742599 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3448633d-41f5-468e-8398-aed436d16d7a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb63b9cd74bf438aa25b72cef8337fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:11:42.743249 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3448633d-41f5-468e-8398-aed436d16d7a service swift] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:11:42.743538 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3448633d-41f5-468e-8398-aed436d16d7a service swift] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:11:42.743818 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3448633d-41f5-468e-8398-aed436d16d7a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:11:42.752129 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3448633d-41f5-468e-8398-aed436d16d7a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:11:42.753703 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3448633d-41f5-468e-8398-aed436d16d7a service swift] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:11:42.761645 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1241/2502] 10.222.0.27 () {60 vars in 1384 bytes} [Fri Jun 10 13:11:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 10 13:11:52.646629 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=33,keystone:DELETE=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:11:52.758503 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:DELETE=5,keystone:SELECT=43,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:12:01.555678 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2556de09-1041-4a16-ac0f-243506fe024d None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:01.556169 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2556de09-1041-4a16-ac0f-243506fe024d None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:01.556345 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2556de09-1041-4a16-ac0f-243506fe024d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:01.574510 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-2556de09-1041-4a16-ac0f-243506fe024d None None] MFA Rules not processed for user `da66362070694e91bbe2cfbbd161b8a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:12:01.575726 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-2556de09-1041-4a16-ac0f-243506fe024d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:12:01.583361 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1262/2503] 10.222.0.27 () {58 vars in 1009 bytes} [Fri Jun 10 13:12:01 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:12:01.596834 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3448633d-41f5-468e-8398-aed436d16d7a service swift] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:01.608313 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3448633d-41f5-468e-8398-aed436d16d7a service swift] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:01.613502 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8add12be-1f81-4ae8-a789-650e054c6443 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb63b9cd74bf438aa25b72cef8337fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:01.614136 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8add12be-1f81-4ae8-a789-650e054c6443 service swift] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:01.614419 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8add12be-1f81-4ae8-a789-650e054c6443 service swift] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:01.614694 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8add12be-1f81-4ae8-a789-650e054c6443 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:01.622850 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8add12be-1f81-4ae8-a789-650e054c6443 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:01.624470 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8add12be-1f81-4ae8-a789-650e054c6443 service swift] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:01.632756 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1242/2504] 10.222.0.27 () {60 vars in 1384 bytes} [Fri Jun 10 13:12:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 10 13:12:06.224218 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994c2a92-c4d4-4038-938f-d1451fc92293 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:06.226656 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994c2a92-c4d4-4038-938f-d1451fc92293 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:06.226656 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994c2a92-c4d4-4038-938f-d1451fc92293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:06.246697 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-994c2a92-c4d4-4038-938f-d1451fc92293 None None] MFA Rules not processed for user `da66362070694e91bbe2cfbbd161b8a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:12:06.248119 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-994c2a92-c4d4-4038-938f-d1451fc92293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:12:06.255608 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1263/2505] 10.222.0.27 () {58 vars in 967 bytes} [Fri Jun 10 13:12:06 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 10 13:12:06.271383 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8add12be-1f81-4ae8-a789-650e054c6443 service swift] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:06.284246 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8add12be-1f81-4ae8-a789-650e054c6443 service swift] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:06.291853 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943110ac-0c84-44ab-b1c7-48360c0aa864 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb63b9cd74bf438aa25b72cef8337fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:06.291853 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943110ac-0c84-44ab-b1c7-48360c0aa864 service swift] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:06.291853 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943110ac-0c84-44ab-b1c7-48360c0aa864 service swift] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:06.291853 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943110ac-0c84-44ab-b1c7-48360c0aa864 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:06.305369 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943110ac-0c84-44ab-b1c7-48360c0aa864 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:06.307567 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943110ac-0c84-44ab-b1c7-48360c0aa864 service swift] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:06.317836 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1243/2506] 10.222.0.27 () {60 vars in 1384 bytes} [Fri Jun 10 13:12:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 10 13:12:16.239555 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=6 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:12:16.313614 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=12 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:12:19.132478 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8da4d5-4821-4417-a48a-ffecd25b7d56 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:19.132932 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8da4d5-4821-4417-a48a-ffecd25b7d56 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:19.133042 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8da4d5-4821-4417-a48a-ffecd25b7d56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:19.151013 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-1e8da4d5-4821-4417-a48a-ffecd25b7d56 None None] MFA Rules not processed for user `da66362070694e91bbe2cfbbd161b8a7`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:12:19.152308 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1e8da4d5-4821-4417-a48a-ffecd25b7d56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:12:19.159581 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1264/2507] 10.222.0.27 () {58 vars in 967 bytes} [Fri Jun 10 13:12:19 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:12:19.171328 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943110ac-0c84-44ab-b1c7-48360c0aa864 service swift] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:19.181667 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943110ac-0c84-44ab-b1c7-48360c0aa864 service swift] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:19.187094 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78562113-1b78-404a-926c-6ad1a86966ca service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb63b9cd74bf438aa25b72cef8337fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:19.187724 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78562113-1b78-404a-926c-6ad1a86966ca service swift] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:19.188047 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78562113-1b78-404a-926c-6ad1a86966ca service swift] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:19.188329 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78562113-1b78-404a-926c-6ad1a86966ca service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:19.196253 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78562113-1b78-404a-926c-6ad1a86966ca service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:19.197827 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78562113-1b78-404a-926c-6ad1a86966ca service swift] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:19.206542 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1244/2508] 10.222.0.27 () {60 vars in 1384 bytes} [Fri Jun 10 13:12:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 10 13:12:20.646068 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8da4d5-4821-4417-a48a-ffecd25b7d56 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:20.647471 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1e8da4d5-4821-4417-a48a-ffecd25b7d56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:12:20.703432 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8da4d5-4821-4417-a48a-ffecd25b7d56 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:20.707037 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0d5235-1388-4cb6-84e6-d6d7609ae09f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:20.707677 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0d5235-1388-4cb6-84e6-d6d7609ae09f service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:20.707990 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0d5235-1388-4cb6-84e6-d6d7609ae09f service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:20.708274 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0d5235-1388-4cb6-84e6-d6d7609ae09f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:20.709996 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1a0d5235-1388-4cb6-84e6-d6d7609ae09f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:12:20.771522 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0d5235-1388-4cb6-84e6-d6d7609ae09f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:20.772992 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0d5235-1388-4cb6-84e6-d6d7609ae09f service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:20.782825 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1265/2509] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:12:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:12:21.973737 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78562113-1b78-404a-926c-6ad1a86966ca service swift] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:21.975741 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-78562113-1b78-404a-926c-6ad1a86966ca service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:12:21.990317 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78562113-1b78-404a-926c-6ad1a86966ca service swift] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:21.994918 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f88aa50-9b5e-4da4-91a6-6aac157a54cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:21.995782 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f88aa50-9b5e-4da4-91a6-6aac157a54cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:21.996109 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f88aa50-9b5e-4da4-91a6-6aac157a54cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:21.996507 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f88aa50-9b5e-4da4-91a6-6aac157a54cc admin admin] PATH_INFO: `/v3/users/24a6e71e55fc4b4db23689d514b3b879` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:21.997754 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f88aa50-9b5e-4da4-91a6-6aac157a54cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=24a6e71e55fc4b4db23689d514b3b879)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:21.999690 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f88aa50-9b5e-4da4-91a6-6aac157a54cc admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:22.212756 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-5f88aa50-9b5e-4da4-91a6-6aac157a54cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24a6e71e55fc4b4db23689d514b3b879'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:22.222248 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-5f88aa50-9b5e-4da4-91a6-6aac157a54cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24a6e71e55fc4b4db23689d514b3b879'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:22.285242 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-5f88aa50-9b5e-4da4-91a6-6aac157a54cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24a6e71e55fc4b4db23689d514b3b879'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:22.286236 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-5f88aa50-9b5e-4da4-91a6-6aac157a54cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24a6e71e55fc4b4db23689d514b3b879'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:22.309865 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-5f88aa50-9b5e-4da4-91a6-6aac157a54cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24a6e71e55fc4b4db23689d514b3b879'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:22.321342 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1245/2510] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:12:21 2022] DELETE /identity/v3/users/24a6e71e55fc4b4db23689d514b3b879 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:12:22.595162 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0d5235-1388-4cb6-84e6-d6d7609ae09f service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:22.606283 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0d5235-1388-4cb6-84e6-d6d7609ae09f service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:22.681935 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0862879-b460-4a6c-9fc5-20d221d9a1a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:22.683398 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0862879-b460-4a6c-9fc5-20d221d9a1a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:22.683679 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0862879-b460-4a6c-9fc5-20d221d9a1a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:22.683988 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0862879-b460-4a6c-9fc5-20d221d9a1a5 admin admin] PATH_INFO: `/v3/projects/0ce61ea5719a40eeb77dc557a849fd82` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:22.685186 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0862879-b460-4a6c-9fc5-20d221d9a1a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ce61ea5719a40eeb77dc557a849fd82)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:22.687074 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0862879-b460-4a6c-9fc5-20d221d9a1a5 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:22.925469 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:12:22.928665 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c0862879-b460-4a6c-9fc5-20d221d9a1a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ce61ea5719a40eeb77dc557a849fd82', 'request_id': 'req-c0862879-b460-4a6c-9fc5-20d221d9a1a5'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:22.936329 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c0862879-b460-4a6c-9fc5-20d221d9a1a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ce61ea5719a40eeb77dc557a849fd82', 'request_id': 'req-c0862879-b460-4a6c-9fc5-20d221d9a1a5'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:22.944767 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c0862879-b460-4a6c-9fc5-20d221d9a1a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ce61ea5719a40eeb77dc557a849fd82', 'request_id': 'req-c0862879-b460-4a6c-9fc5-20d221d9a1a5'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:23.027556 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c0862879-b460-4a6c-9fc5-20d221d9a1a5 admin admin] The token cache is being invalidate because project 0ce61ea5719a40eeb77dc557a849fd82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:12:23.027848 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c0862879-b460-4a6c-9fc5-20d221d9a1a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:23.028499 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c0862879-b460-4a6c-9fc5-20d221d9a1a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:23.030299 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1266/2511] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:12:22 2022] DELETE /identity/v3/projects/0ce61ea5719a40eeb77dc557a849fd82 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:12:24.684671 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f88aa50-9b5e-4da4-91a6-6aac157a54cc admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:24.686123 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5f88aa50-9b5e-4da4-91a6-6aac157a54cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:12:24.700166 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f88aa50-9b5e-4da4-91a6-6aac157a54cc admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:24.755977 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f37ffe-3327-4dbf-a9db-fce80c2716a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:24.756885 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f37ffe-3327-4dbf-a9db-fce80c2716a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:24.757231 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f37ffe-3327-4dbf-a9db-fce80c2716a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:24.757511 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f37ffe-3327-4dbf-a9db-fce80c2716a6 admin admin] PATH_INFO: `/v3/users/634e6116f688494c9e11d59479f94a43` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:24.758711 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f37ffe-3327-4dbf-a9db-fce80c2716a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=634e6116f688494c9e11d59479f94a43)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:24.760669 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f37ffe-3327-4dbf-a9db-fce80c2716a6 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:24.922042 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-95f37ffe-3327-4dbf-a9db-fce80c2716a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '634e6116f688494c9e11d59479f94a43'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:24.929456 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-95f37ffe-3327-4dbf-a9db-fce80c2716a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '634e6116f688494c9e11d59479f94a43'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:24.934091 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:12:25.004820 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-95f37ffe-3327-4dbf-a9db-fce80c2716a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '634e6116f688494c9e11d59479f94a43'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:25.005431 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-95f37ffe-3327-4dbf-a9db-fce80c2716a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '634e6116f688494c9e11d59479f94a43'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:25.019320 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-95f37ffe-3327-4dbf-a9db-fce80c2716a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '634e6116f688494c9e11d59479f94a43'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:25.025804 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1246/2512] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:12:24 2022] DELETE /identity/v3/users/634e6116f688494c9e11d59479f94a43 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:12:25.275037 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0862879-b460-4a6c-9fc5-20d221d9a1a5 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:25.277037 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c0862879-b460-4a6c-9fc5-20d221d9a1a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:12:25.290205 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0862879-b460-4a6c-9fc5-20d221d9a1a5 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:25.350369 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4bef7d-b968-4c67-98c3-69f2da6883a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:25.351226 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4bef7d-b968-4c67-98c3-69f2da6883a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:25.351514 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4bef7d-b968-4c67-98c3-69f2da6883a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:25.351793 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4bef7d-b968-4c67-98c3-69f2da6883a5 admin admin] PATH_INFO: `/v3/projects/1a45808d686e417ab6e8df91aa2735a7` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:25.353074 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4bef7d-b968-4c67-98c3-69f2da6883a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a45808d686e417ab6e8df91aa2735a7)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:25.355761 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4bef7d-b968-4c67-98c3-69f2da6883a5 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:25.444841 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-bd4bef7d-b968-4c67-98c3-69f2da6883a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a45808d686e417ab6e8df91aa2735a7', 'request_id': 'req-bd4bef7d-b968-4c67-98c3-69f2da6883a5'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:25.450897 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-bd4bef7d-b968-4c67-98c3-69f2da6883a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a45808d686e417ab6e8df91aa2735a7', 'request_id': 'req-bd4bef7d-b968-4c67-98c3-69f2da6883a5'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:25.456951 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-bd4bef7d-b968-4c67-98c3-69f2da6883a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a45808d686e417ab6e8df91aa2735a7', 'request_id': 'req-bd4bef7d-b968-4c67-98c3-69f2da6883a5'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:25.524178 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-bd4bef7d-b968-4c67-98c3-69f2da6883a5 admin admin] The token cache is being invalidate because project 1a45808d686e417ab6e8df91aa2735a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:12:25.524469 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-bd4bef7d-b968-4c67-98c3-69f2da6883a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:25.525096 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-bd4bef7d-b968-4c67-98c3-69f2da6883a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:25.527321 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1267/2513] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:12:25 2022] DELETE /identity/v3/projects/1a45808d686e417ab6e8df91aa2735a7 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:12:25.532425 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0096fb1-738a-49f3-a7b0-7dbb3af9922c None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:25.532720 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0096fb1-738a-49f3-a7b0-7dbb3af9922c None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:25.533002 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0096fb1-738a-49f3-a7b0-7dbb3af9922c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:25.550314 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-b0096fb1-738a-49f3-a7b0-7dbb3af9922c None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:12:25.551509 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b0096fb1-738a-49f3-a7b0-7dbb3af9922c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:12:25.609594 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1247/2514] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:12:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:12:25.617468 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3447028-3eef-47f1-8aa2-e46c28011561 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:25.617759 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3447028-3eef-47f1-8aa2-e46c28011561 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:25.618040 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3447028-3eef-47f1-8aa2-e46c28011561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:25.636937 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-c3447028-3eef-47f1-8aa2-e46c28011561 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:12:25.638840 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c3447028-3eef-47f1-8aa2-e46c28011561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:12:25.646247 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1268/2515] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:12:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:12:25.650387 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0096fb1-738a-49f3-a7b0-7dbb3af9922c None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:25.660994 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0096fb1-738a-49f3-a7b0-7dbb3af9922c None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:25.665920 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348c458e-1579-4282-b108-4f58cd6f8f65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:25.675730 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348c458e-1579-4282-b108-4f58cd6f8f65 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:25.675730 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348c458e-1579-4282-b108-4f58cd6f8f65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:25.675730 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348c458e-1579-4282-b108-4f58cd6f8f65 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:25.675730 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348c458e-1579-4282-b108-4f58cd6f8f65 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 13:12:25.675730 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348c458e-1579-4282-b108-4f58cd6f8f65 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:25.675730 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348c458e-1579-4282-b108-4f58cd6f8f65 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:25.681676 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1248/2516] 10.222.0.27 () {58 vars in 1152 bytes} [Fri Jun 10 13:12:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 13:12:25.685847 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3447028-3eef-47f1-8aa2-e46c28011561 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:25.697094 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3447028-3eef-47f1-8aa2-e46c28011561 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:25.702331 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf74cbe3-8eb4-402e-bd4d-21ce97c072b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:25.702930 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf74cbe3-8eb4-402e-bd4d-21ce97c072b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:25.703214 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf74cbe3-8eb4-402e-bd4d-21ce97c072b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:25.703489 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf74cbe3-8eb4-402e-bd4d-21ce97c072b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:25.704209 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf74cbe3-8eb4-402e-bd4d-21ce97c072b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:25.706084 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf74cbe3-8eb4-402e-bd4d-21ce97c072b0 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:25.801632 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1269/2517] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:12:25 2022] POST /identity/v3/projects => generated 367 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:12:25.805734 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348c458e-1579-4282-b108-4f58cd6f8f65 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:25.816362 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348c458e-1579-4282-b108-4f58cd6f8f65 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:25.875034 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515d4906-581e-4054-892a-4e72a8897933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:25.875711 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515d4906-581e-4054-892a-4e72a8897933 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:25.876041 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515d4906-581e-4054-892a-4e72a8897933 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:25.876325 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515d4906-581e-4054-892a-4e72a8897933 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:25.877026 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515d4906-581e-4054-892a-4e72a8897933 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:25.879098 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515d4906-581e-4054-892a-4e72a8897933 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:25.954271 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1249/2518] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:12:25 2022] POST /identity/v3/users => generated 284 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:12:25.958111 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf74cbe3-8eb4-402e-bd4d-21ce97c072b0 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:25.968280 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf74cbe3-8eb4-402e-bd4d-21ce97c072b0 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:25.973094 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9654fb3b-a21e-41d7-b515-2a99e136cd25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:25.973748 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9654fb3b-a21e-41d7-b515-2a99e136cd25 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:25.974030 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9654fb3b-a21e-41d7-b515-2a99e136cd25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:25.974309 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9654fb3b-a21e-41d7-b515-2a99e136cd25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:25.975107 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9654fb3b-a21e-41d7-b515-2a99e136cd25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:25.976729 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9654fb3b-a21e-41d7-b515-2a99e136cd25 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:25.987081 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1270/2519] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:12:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:12:25.994442 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515d4906-581e-4054-892a-4e72a8897933 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:26.004831 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515d4906-581e-4054-892a-4e72a8897933 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:26.009395 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef19aef-7234-49dd-a146-94d9ff5082b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:26.010097 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef19aef-7234-49dd-a146-94d9ff5082b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:26.010377 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef19aef-7234-49dd-a146-94d9ff5082b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:26.010652 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef19aef-7234-49dd-a146-94d9ff5082b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:26.012043 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef19aef-7234-49dd-a146-94d9ff5082b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:26.013625 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef19aef-7234-49dd-a146-94d9ff5082b7 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:26.024423 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1250/2520] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:12:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:12:26.028412 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9654fb3b-a21e-41d7-b515-2a99e136cd25 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:26.039718 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9654fb3b-a21e-41d7-b515-2a99e136cd25 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:26.044483 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5921a8ac-c059-4386-917d-a8ce7de9d962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:26.045364 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5921a8ac-c059-4386-917d-a8ce7de9d962 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:26.045651 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5921a8ac-c059-4386-917d-a8ce7de9d962 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:26.045931 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5921a8ac-c059-4386-917d-a8ce7de9d962 admin admin] PATH_INFO: `/v3/projects/901d0a139be24f6eb4ea01c63dbf84ca/users/bf4dbbac9f654c31bbb0934cc677fd22/roles/43106c664e1b40de984c0c62701149c8` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:26.061017 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5921a8ac-c059-4386-917d-a8ce7de9d962 admin admin] RBAC: Authorizing `identity:create_grant(project_id=901d0a139be24f6eb4ea01c63dbf84ca, user_id=bf4dbbac9f654c31bbb0934cc677fd22, role_id=43106c664e1b40de984c0c62701149c8)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:26.063668 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5921a8ac-c059-4386-917d-a8ce7de9d962 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:26.132785 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:12:26.138094 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1271/2521] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:12:26 2022] PUT /identity/v3/projects/901d0a139be24f6eb4ea01c63dbf84ca/users/bf4dbbac9f654c31bbb0934cc677fd22/roles/43106c664e1b40de984c0c62701149c8 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 10 13:12:26.148781 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ca3753-77c3-4599-8f8f-42b5d871f5ea None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:26.149092 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ca3753-77c3-4599-8f8f-42b5d871f5ea None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:26.149376 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ca3753-77c3-4599-8f8f-42b5d871f5ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:26.176279 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-25ca3753-77c3-4599-8f8f-42b5d871f5ea None None] MFA Rules not processed for user `bf4dbbac9f654c31bbb0934cc677fd22`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:12:26.177470 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-25ca3753-77c3-4599-8f8f-42b5d871f5ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:12:26.210565 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:12:26.244684 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1251/2522] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:12:26 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:12:26.249797 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5921a8ac-c059-4386-917d-a8ce7de9d962 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:26.311434 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5921a8ac-c059-4386-917d-a8ce7de9d962 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:26.316102 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2795daf-b7b0-426c-a7fc-e27e7f2cc3e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:26.316752 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2795daf-b7b0-426c-a7fc-e27e7f2cc3e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:26.317035 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2795daf-b7b0-426c-a7fc-e27e7f2cc3e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:26.317309 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2795daf-b7b0-426c-a7fc-e27e7f2cc3e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:26.317993 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2795daf-b7b0-426c-a7fc-e27e7f2cc3e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:26.319876 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2795daf-b7b0-426c-a7fc-e27e7f2cc3e9 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:26.406997 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1272/2523] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:12:26 2022] POST /identity/v3/projects => generated 367 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:12:26.409811 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ca3753-77c3-4599-8f8f-42b5d871f5ea None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:26.420819 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ca3753-77c3-4599-8f8f-42b5d871f5ea None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:26.476058 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e473b0-9b9d-4e2c-80ce-a33a53e4f9d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:26.476610 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e473b0-9b9d-4e2c-80ce-a33a53e4f9d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:26.476895 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e473b0-9b9d-4e2c-80ce-a33a53e4f9d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:26.477167 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e473b0-9b9d-4e2c-80ce-a33a53e4f9d5 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:26.477845 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e473b0-9b9d-4e2c-80ce-a33a53e4f9d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:26.479741 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e473b0-9b9d-4e2c-80ce-a33a53e4f9d5 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:26.593651 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1252/2524] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:12:26 2022] POST /identity/v3/users => generated 290 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 10 13:12:26.597571 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2795daf-b7b0-426c-a7fc-e27e7f2cc3e9 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:26.608439 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2795daf-b7b0-426c-a7fc-e27e7f2cc3e9 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:26.612958 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ce5b2d-211b-45a1-8fdc-932c15b82508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:26.613612 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ce5b2d-211b-45a1-8fdc-932c15b82508 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:26.613896 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ce5b2d-211b-45a1-8fdc-932c15b82508 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:26.614174 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ce5b2d-211b-45a1-8fdc-932c15b82508 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:26.614898 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ce5b2d-211b-45a1-8fdc-932c15b82508 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:26.616520 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ce5b2d-211b-45a1-8fdc-932c15b82508 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:26.627078 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1273/2525] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:12:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:12:26.631005 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e473b0-9b9d-4e2c-80ce-a33a53e4f9d5 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:26.640889 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e473b0-9b9d-4e2c-80ce-a33a53e4f9d5 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:26.645491 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2776b7-4037-4e6e-849b-71cacdda8374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:26.646360 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2776b7-4037-4e6e-849b-71cacdda8374 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:26.646646 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2776b7-4037-4e6e-849b-71cacdda8374 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:26.646923 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2776b7-4037-4e6e-849b-71cacdda8374 admin admin] PATH_INFO: `/v3/projects/8c677dad2c6b4e828f419092903c42b7/users/9943413f14cc48588560fed95eaf0446/roles/b43d5a8247e244869c2c70f986f42e8a` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:26.662280 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2776b7-4037-4e6e-849b-71cacdda8374 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c677dad2c6b4e828f419092903c42b7, user_id=9943413f14cc48588560fed95eaf0446, role_id=b43d5a8247e244869c2c70f986f42e8a)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:26.665000 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2776b7-4037-4e6e-849b-71cacdda8374 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:26.742530 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1253/2526] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:12:26 2022] PUT /identity/v3/projects/8c677dad2c6b4e828f419092903c42b7/users/9943413f14cc48588560fed95eaf0446/roles/b43d5a8247e244869c2c70f986f42e8a => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:12:26.746371 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ce5b2d-211b-45a1-8fdc-932c15b82508 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:26.756539 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ce5b2d-211b-45a1-8fdc-932c15b82508 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:26.816520 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d09f93-59f6-40d6-af81-663eb03e08fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:26.817326 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d09f93-59f6-40d6-af81-663eb03e08fa admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:26.817616 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d09f93-59f6-40d6-af81-663eb03e08fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:26.817894 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d09f93-59f6-40d6-af81-663eb03e08fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:26.818607 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d09f93-59f6-40d6-af81-663eb03e08fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:26.820227 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d09f93-59f6-40d6-af81-663eb03e08fa admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:26.829794 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1274/2527] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:12:26 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:12:26.834040 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2776b7-4037-4e6e-849b-71cacdda8374 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:26.844782 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2776b7-4037-4e6e-849b-71cacdda8374 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:26.849403 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3341928a-5e60-4c94-ad67-03ec05101994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:26.850265 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3341928a-5e60-4c94-ad67-03ec05101994 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:26.850542 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3341928a-5e60-4c94-ad67-03ec05101994 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:26.850815 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3341928a-5e60-4c94-ad67-03ec05101994 admin admin] PATH_INFO: `/v3/projects/8c677dad2c6b4e828f419092903c42b7/users/9943413f14cc48588560fed95eaf0446/roles/b43d5a8247e244869c2c70f986f42e8a` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:26.852851 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3341928a-5e60-4c94-ad67-03ec05101994 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c677dad2c6b4e828f419092903c42b7, user_id=9943413f14cc48588560fed95eaf0446, role_id=b43d5a8247e244869c2c70f986f42e8a)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:26.856254 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3341928a-5e60-4c94-ad67-03ec05101994 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:26.866738 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1254/2528] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:12:26 2022] PUT /identity/v3/projects/8c677dad2c6b4e828f419092903c42b7/users/9943413f14cc48588560fed95eaf0446/roles/b43d5a8247e244869c2c70f986f42e8a => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:12:26.876637 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988fe420-ec10-4f7a-b844-d3f0983eacc7 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:26.876930 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988fe420-ec10-4f7a-b844-d3f0983eacc7 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:26.877232 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988fe420-ec10-4f7a-b844-d3f0983eacc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:26.904033 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-988fe420-ec10-4f7a-b844-d3f0983eacc7 None None] MFA Rules not processed for user `9943413f14cc48588560fed95eaf0446`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:12:26.905481 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-988fe420-ec10-4f7a-b844-d3f0983eacc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:12:26.961527 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:12:26.971520 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1275/2529] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:12:26 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:12:26.982742 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3341928a-5e60-4c94-ad67-03ec05101994 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:26.984690 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-3341928a-5e60-4c94-ad67-03ec05101994 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:12:27.041077 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3341928a-5e60-4c94-ad67-03ec05101994 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:27.044676 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c956a492-df79-48db-82a4-72430d627b87 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6cc95a563b941ac92ab283aa0d5a5d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:27.045310 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c956a492-df79-48db-82a4-72430d627b87 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:27.045591 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c956a492-df79-48db-82a4-72430d627b87 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:27.045870 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c956a492-df79-48db-82a4-72430d627b87 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:27.054633 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c956a492-df79-48db-82a4-72430d627b87 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:27.056072 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c956a492-df79-48db-82a4-72430d627b87 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:27.065234 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1255/2530] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:12:26 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:12:27.145648 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988fe420-ec10-4f7a-b844-d3f0983eacc7 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:27.147202 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-988fe420-ec10-4f7a-b844-d3f0983eacc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:12:27.160666 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988fe420-ec10-4f7a-b844-d3f0983eacc7 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:27.164715 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05e9e7a-e816-40d8-bdfc-85a9d1fde38b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6cc95a563b941ac92ab283aa0d5a5d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:27.165359 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05e9e7a-e816-40d8-bdfc-85a9d1fde38b service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:27.165639 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05e9e7a-e816-40d8-bdfc-85a9d1fde38b service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:27.165915 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05e9e7a-e816-40d8-bdfc-85a9d1fde38b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:27.224233 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05e9e7a-e816-40d8-bdfc-85a9d1fde38b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:27.225668 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05e9e7a-e816-40d8-bdfc-85a9d1fde38b service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:27.234559 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1276/2531] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:12:27 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:12:31.717414 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c956a492-df79-48db-82a4-72430d627b87 service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:31.748348 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:12:31.779730 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c956a492-df79-48db-82a4-72430d627b87 service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:31.784559 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88974df-4bda-4bc9-9459-c6c2e97db9a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:31.785419 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88974df-4bda-4bc9-9459-c6c2e97db9a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:31.785706 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88974df-4bda-4bc9-9459-c6c2e97db9a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:31.786033 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88974df-4bda-4bc9-9459-c6c2e97db9a6 admin admin] PATH_INFO: `/v3/users/bf4dbbac9f654c31bbb0934cc677fd22` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:31.787208 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88974df-4bda-4bc9-9459-c6c2e97db9a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf4dbbac9f654c31bbb0934cc677fd22)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:31.789160 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88974df-4bda-4bc9-9459-c6c2e97db9a6 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:31.937350 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-d88974df-4bda-4bc9-9459-c6c2e97db9a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf4dbbac9f654c31bbb0934cc677fd22'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:31.944489 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-d88974df-4bda-4bc9-9459-c6c2e97db9a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf4dbbac9f654c31bbb0934cc677fd22'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:32.044826 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-d88974df-4bda-4bc9-9459-c6c2e97db9a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf4dbbac9f654c31bbb0934cc677fd22'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:32.045460 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-d88974df-4bda-4bc9-9459-c6c2e97db9a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf4dbbac9f654c31bbb0934cc677fd22'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:32.057881 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-d88974df-4bda-4bc9-9459-c6c2e97db9a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf4dbbac9f654c31bbb0934cc677fd22'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:32.064535 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1256/2532] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:12:31 2022] DELETE /identity/v3/users/bf4dbbac9f654c31bbb0934cc677fd22 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:12:32.069996 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409c688f-a0c2-4feb-b705-40fc2569b3f6 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:32.080286 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409c688f-a0c2-4feb-b705-40fc2569b3f6 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:32.080286 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409c688f-a0c2-4feb-b705-40fc2569b3f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:32.087648 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-409c688f-a0c2-4feb-b705-40fc2569b3f6 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:12:32.088938 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-409c688f-a0c2-4feb-b705-40fc2569b3f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:12:32.148612 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1277/2533] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:12:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:12:32.158355 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88974df-4bda-4bc9-9459-c6c2e97db9a6 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:32.159748 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d88974df-4bda-4bc9-9459-c6c2e97db9a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:12:32.215958 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88974df-4bda-4bc9-9459-c6c2e97db9a6 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:32.219454 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24db6ddc-e352-4385-9bf6-4083f1cda5a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:32.220114 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24db6ddc-e352-4385-9bf6-4083f1cda5a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:32.220422 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24db6ddc-e352-4385-9bf6-4083f1cda5a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:32.220698 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24db6ddc-e352-4385-9bf6-4083f1cda5a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:32.229505 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24db6ddc-e352-4385-9bf6-4083f1cda5a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:32.231435 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24db6ddc-e352-4385-9bf6-4083f1cda5a3 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:32.241422 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1257/2534] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:12:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:12:32.624270 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409c688f-a0c2-4feb-b705-40fc2569b3f6 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:32.625706 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-409c688f-a0c2-4feb-b705-40fc2569b3f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:12:32.637521 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409c688f-a0c2-4feb-b705-40fc2569b3f6 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:32.642352 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969f3fd1-168a-4004-adac-4074e5cd5c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:32.643143 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969f3fd1-168a-4004-adac-4074e5cd5c69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:32.643429 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969f3fd1-168a-4004-adac-4074e5cd5c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:32.644300 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969f3fd1-168a-4004-adac-4074e5cd5c69 admin admin] PATH_INFO: `/v3/projects/901d0a139be24f6eb4ea01c63dbf84ca` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:32.645146 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969f3fd1-168a-4004-adac-4074e5cd5c69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=901d0a139be24f6eb4ea01c63dbf84ca)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:32.647037 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969f3fd1-168a-4004-adac-4074e5cd5c69 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:32.748942 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-969f3fd1-168a-4004-adac-4074e5cd5c69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '901d0a139be24f6eb4ea01c63dbf84ca', 'request_id': 'req-969f3fd1-168a-4004-adac-4074e5cd5c69'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:32.755548 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-969f3fd1-168a-4004-adac-4074e5cd5c69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '901d0a139be24f6eb4ea01c63dbf84ca', 'request_id': 'req-969f3fd1-168a-4004-adac-4074e5cd5c69'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:32.761871 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-969f3fd1-168a-4004-adac-4074e5cd5c69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '901d0a139be24f6eb4ea01c63dbf84ca', 'request_id': 'req-969f3fd1-168a-4004-adac-4074e5cd5c69'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:32.832725 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-969f3fd1-168a-4004-adac-4074e5cd5c69 admin admin] The token cache is being invalidate because project 901d0a139be24f6eb4ea01c63dbf84ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:12:32.833031 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-969f3fd1-168a-4004-adac-4074e5cd5c69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:32.833658 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-969f3fd1-168a-4004-adac-4074e5cd5c69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:32.835228 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1278/2535] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:12:32 2022] DELETE /identity/v3/projects/901d0a139be24f6eb4ea01c63dbf84ca => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:12:32.839764 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24db6ddc-e352-4385-9bf6-4083f1cda5a3 service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:32.849786 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24db6ddc-e352-4385-9bf6-4083f1cda5a3 service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:32.899500 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:12:32.909839 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac5e6c5-a548-4d26-90b7-1f5e8e5cf423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:32.910717 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac5e6c5-a548-4d26-90b7-1f5e8e5cf423 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:32.911001 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac5e6c5-a548-4d26-90b7-1f5e8e5cf423 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:32.911801 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac5e6c5-a548-4d26-90b7-1f5e8e5cf423 admin admin] PATH_INFO: `/v3/users/9943413f14cc48588560fed95eaf0446` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:32.912493 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac5e6c5-a548-4d26-90b7-1f5e8e5cf423 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9943413f14cc48588560fed95eaf0446)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:32.914395 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac5e6c5-a548-4d26-90b7-1f5e8e5cf423 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:33.083486 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-aac5e6c5-a548-4d26-90b7-1f5e8e5cf423 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9943413f14cc48588560fed95eaf0446'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:33.091291 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-aac5e6c5-a548-4d26-90b7-1f5e8e5cf423 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9943413f14cc48588560fed95eaf0446'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:33.142219 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-aac5e6c5-a548-4d26-90b7-1f5e8e5cf423 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9943413f14cc48588560fed95eaf0446'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:33.142836 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-aac5e6c5-a548-4d26-90b7-1f5e8e5cf423 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9943413f14cc48588560fed95eaf0446'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:33.157231 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-aac5e6c5-a548-4d26-90b7-1f5e8e5cf423 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9943413f14cc48588560fed95eaf0446'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:33.163531 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1258/2536] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:12:32 2022] DELETE /identity/v3/users/9943413f14cc48588560fed95eaf0446 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:12:33.658665 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969f3fd1-168a-4004-adac-4074e5cd5c69 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:33.660438 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-969f3fd1-168a-4004-adac-4074e5cd5c69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:12:33.690365 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969f3fd1-168a-4004-adac-4074e5cd5c69 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:33.713400 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:12:33.772909 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4869256c-5796-4d8b-bc23-a4ba42b951c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:33.773782 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4869256c-5796-4d8b-bc23-a4ba42b951c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:33.774440 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4869256c-5796-4d8b-bc23-a4ba42b951c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:33.774440 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4869256c-5796-4d8b-bc23-a4ba42b951c8 admin admin] PATH_INFO: `/v3/projects/8c677dad2c6b4e828f419092903c42b7` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:33.775615 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4869256c-5796-4d8b-bc23-a4ba42b951c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c677dad2c6b4e828f419092903c42b7)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:33.777779 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4869256c-5796-4d8b-bc23-a4ba42b951c8 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:33.896386 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4869256c-5796-4d8b-bc23-a4ba42b951c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c677dad2c6b4e828f419092903c42b7', 'request_id': 'req-4869256c-5796-4d8b-bc23-a4ba42b951c8'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:33.901888 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4869256c-5796-4d8b-bc23-a4ba42b951c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c677dad2c6b4e828f419092903c42b7', 'request_id': 'req-4869256c-5796-4d8b-bc23-a4ba42b951c8'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:33.908961 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4869256c-5796-4d8b-bc23-a4ba42b951c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c677dad2c6b4e828f419092903c42b7', 'request_id': 'req-4869256c-5796-4d8b-bc23-a4ba42b951c8'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:33.987168 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4869256c-5796-4d8b-bc23-a4ba42b951c8 admin admin] The token cache is being invalidate because project 8c677dad2c6b4e828f419092903c42b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:12:33.987462 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4869256c-5796-4d8b-bc23-a4ba42b951c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:33.989569 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4869256c-5796-4d8b-bc23-a4ba42b951c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:33.989824 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1279/2537] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:12:33 2022] DELETE /identity/v3/projects/8c677dad2c6b4e828f419092903c42b7 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:12:33.994826 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2652ec-668f-4ff4-acca-55fa1f1bc4d9 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:33.995340 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2652ec-668f-4ff4-acca-55fa1f1bc4d9 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:33.996195 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2652ec-668f-4ff4-acca-55fa1f1bc4d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:34.014440 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-1b2652ec-668f-4ff4-acca-55fa1f1bc4d9 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:12:34.015565 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-1b2652ec-668f-4ff4-acca-55fa1f1bc4d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:12:34.087187 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1259/2538] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:12:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:12:34.095503 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fe55f7-2d63-47bb-9388-188f05cb813c None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:34.101319 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fe55f7-2d63-47bb-9388-188f05cb813c None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:34.101319 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fe55f7-2d63-47bb-9388-188f05cb813c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:34.114922 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-53fe55f7-2d63-47bb-9388-188f05cb813c None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:12:34.116415 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-53fe55f7-2d63-47bb-9388-188f05cb813c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:12:34.124749 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1280/2539] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:12:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:12:34.128798 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2652ec-668f-4ff4-acca-55fa1f1bc4d9 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:34.139239 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2652ec-668f-4ff4-acca-55fa1f1bc4d9 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:34.144409 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcadc227-a0bd-47f4-abee-110e75a7a298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:34.145066 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcadc227-a0bd-47f4-abee-110e75a7a298 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:34.145354 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcadc227-a0bd-47f4-abee-110e75a7a298 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:34.145634 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcadc227-a0bd-47f4-abee-110e75a7a298 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:34.146371 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcadc227-a0bd-47f4-abee-110e75a7a298 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 13:12:34.146748 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcadc227-a0bd-47f4-abee-110e75a7a298 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:34.148409 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcadc227-a0bd-47f4-abee-110e75a7a298 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:34.161196 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1260/2540] 10.222.0.27 () {58 vars in 1152 bytes} [Fri Jun 10 13:12:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 13:12:34.165642 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fe55f7-2d63-47bb-9388-188f05cb813c None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:34.175764 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fe55f7-2d63-47bb-9388-188f05cb813c None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:34.180548 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858b8ace-5bd7-4a92-a955-f62d10dbf225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:34.181204 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858b8ace-5bd7-4a92-a955-f62d10dbf225 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:34.181488 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858b8ace-5bd7-4a92-a955-f62d10dbf225 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:34.181762 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858b8ace-5bd7-4a92-a955-f62d10dbf225 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:34.182447 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858b8ace-5bd7-4a92-a955-f62d10dbf225 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:34.184394 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858b8ace-5bd7-4a92-a955-f62d10dbf225 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:34.310176 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1281/2541] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:12:34 2022] POST /identity/v3/projects => generated 379 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:12:34.314339 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcadc227-a0bd-47f4-abee-110e75a7a298 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:34.326321 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcadc227-a0bd-47f4-abee-110e75a7a298 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:34.392742 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:12:34.400352 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fce12a2-f3e0-4dd5-ae1a-bf46f0991eaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:34.401044 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fce12a2-f3e0-4dd5-ae1a-bf46f0991eaf admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:34.401597 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fce12a2-f3e0-4dd5-ae1a-bf46f0991eaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:34.402388 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fce12a2-f3e0-4dd5-ae1a-bf46f0991eaf admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:34.403086 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fce12a2-f3e0-4dd5-ae1a-bf46f0991eaf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:34.405578 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fce12a2-f3e0-4dd5-ae1a-bf46f0991eaf admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:34.494326 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1261/2542] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:12:34 2022] POST /identity/v3/users => generated 290 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:12:34.495576 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858b8ace-5bd7-4a92-a955-f62d10dbf225 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:34.506586 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858b8ace-5bd7-4a92-a955-f62d10dbf225 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:34.512170 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76efb85-a18b-4441-96ef-73861c62d669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:34.512839 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76efb85-a18b-4441-96ef-73861c62d669 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:34.518241 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76efb85-a18b-4441-96ef-73861c62d669 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:34.518241 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76efb85-a18b-4441-96ef-73861c62d669 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:34.518241 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76efb85-a18b-4441-96ef-73861c62d669 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:34.518241 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76efb85-a18b-4441-96ef-73861c62d669 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:34.527433 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1282/2543] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:12:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:12:34.535436 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fce12a2-f3e0-4dd5-ae1a-bf46f0991eaf admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:34.542556 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fce12a2-f3e0-4dd5-ae1a-bf46f0991eaf admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:34.547239 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2d1eeb-7566-49c0-99c2-2c1d31d5b504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:34.547982 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2d1eeb-7566-49c0-99c2-2c1d31d5b504 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:34.550268 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2d1eeb-7566-49c0-99c2-2c1d31d5b504 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:34.550268 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2d1eeb-7566-49c0-99c2-2c1d31d5b504 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:34.550268 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2d1eeb-7566-49c0-99c2-2c1d31d5b504 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:34.550828 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2d1eeb-7566-49c0-99c2-2c1d31d5b504 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:34.561473 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1262/2544] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:12:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:12:34.565700 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76efb85-a18b-4441-96ef-73861c62d669 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:34.576284 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76efb85-a18b-4441-96ef-73861c62d669 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:34.582522 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce986206-a72a-4856-b955-cdc9b9c07526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:34.583410 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce986206-a72a-4856-b955-cdc9b9c07526 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:34.583696 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce986206-a72a-4856-b955-cdc9b9c07526 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:34.584036 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce986206-a72a-4856-b955-cdc9b9c07526 admin admin] PATH_INFO: `/v3/projects/9f1fb2b9bc614686adfdd494db94558b/users/5b5d9e7ffa4749ee9494952940e582b5/roles/43106c664e1b40de984c0c62701149c8` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:34.599716 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce986206-a72a-4856-b955-cdc9b9c07526 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f1fb2b9bc614686adfdd494db94558b, user_id=5b5d9e7ffa4749ee9494952940e582b5, role_id=43106c664e1b40de984c0c62701149c8)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:34.602385 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce986206-a72a-4856-b955-cdc9b9c07526 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:34.673993 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1283/2545] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:12:34 2022] PUT /identity/v3/projects/9f1fb2b9bc614686adfdd494db94558b/users/5b5d9e7ffa4749ee9494952940e582b5/roles/43106c664e1b40de984c0c62701149c8 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:12:34.685170 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8855d9b4-0bfb-4541-bd6e-968e4aaec383 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:34.685459 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8855d9b4-0bfb-4541-bd6e-968e4aaec383 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:34.685735 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8855d9b4-0bfb-4541-bd6e-968e4aaec383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:34.713317 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-8855d9b4-0bfb-4541-bd6e-968e4aaec383 None None] MFA Rules not processed for user `5b5d9e7ffa4749ee9494952940e582b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:12:34.715133 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-8855d9b4-0bfb-4541-bd6e-968e4aaec383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:12:34.781821 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1263/2546] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:12:34 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:12:34.784216 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce986206-a72a-4856-b955-cdc9b9c07526 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:34.829128 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:12:34.844693 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce986206-a72a-4856-b955-cdc9b9c07526 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:34.849348 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d5f9c3-a4f3-48e0-8242-0cb989bfc1f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:34.849998 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d5f9c3-a4f3-48e0-8242-0cb989bfc1f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:34.850283 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d5f9c3-a4f3-48e0-8242-0cb989bfc1f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:34.850558 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d5f9c3-a4f3-48e0-8242-0cb989bfc1f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:34.851228 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d5f9c3-a4f3-48e0-8242-0cb989bfc1f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:34.853154 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d5f9c3-a4f3-48e0-8242-0cb989bfc1f6 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:34.942778 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1284/2547] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:12:34 2022] POST /identity/v3/projects => generated 379 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:12:34.946917 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8855d9b4-0bfb-4541-bd6e-968e4aaec383 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:34.958169 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8855d9b4-0bfb-4541-bd6e-968e4aaec383 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:35.012907 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec89e947-1a85-4eb7-8bfa-78b4efc55f00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:35.013570 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec89e947-1a85-4eb7-8bfa-78b4efc55f00 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:35.013854 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec89e947-1a85-4eb7-8bfa-78b4efc55f00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:35.014132 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec89e947-1a85-4eb7-8bfa-78b4efc55f00 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:35.014826 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec89e947-1a85-4eb7-8bfa-78b4efc55f00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:35.016794 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec89e947-1a85-4eb7-8bfa-78b4efc55f00 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:35.098426 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1264/2548] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:12:34 2022] POST /identity/v3/users => generated 296 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:12:35.099734 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d5f9c3-a4f3-48e0-8242-0cb989bfc1f6 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:35.109976 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d5f9c3-a4f3-48e0-8242-0cb989bfc1f6 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:35.114618 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83286efc-26d3-4d43-af4e-09676284fe88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:35.115301 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83286efc-26d3-4d43-af4e-09676284fe88 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:35.115588 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83286efc-26d3-4d43-af4e-09676284fe88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:35.115886 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83286efc-26d3-4d43-af4e-09676284fe88 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:35.116612 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83286efc-26d3-4d43-af4e-09676284fe88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:35.118208 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83286efc-26d3-4d43-af4e-09676284fe88 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:35.129135 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1285/2549] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:12:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:12:35.133294 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec89e947-1a85-4eb7-8bfa-78b4efc55f00 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:35.143961 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec89e947-1a85-4eb7-8bfa-78b4efc55f00 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:35.148830 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381e890e-79b4-4c17-a4f1-c02fcbbbf6ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:35.149703 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381e890e-79b4-4c17-a4f1-c02fcbbbf6ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:35.150017 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381e890e-79b4-4c17-a4f1-c02fcbbbf6ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:35.150325 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381e890e-79b4-4c17-a4f1-c02fcbbbf6ee admin admin] PATH_INFO: `/v3/projects/d29798d941fa4b168bee64e2b9f029a3/users/50a36e23d43d433ba261781b1d6de632/roles/b43d5a8247e244869c2c70f986f42e8a` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:35.165233 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381e890e-79b4-4c17-a4f1-c02fcbbbf6ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=d29798d941fa4b168bee64e2b9f029a3, user_id=50a36e23d43d433ba261781b1d6de632, role_id=b43d5a8247e244869c2c70f986f42e8a)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:35.168430 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381e890e-79b4-4c17-a4f1-c02fcbbbf6ee admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:35.233341 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1265/2550] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:12:35 2022] PUT /identity/v3/projects/d29798d941fa4b168bee64e2b9f029a3/users/50a36e23d43d433ba261781b1d6de632/roles/b43d5a8247e244869c2c70f986f42e8a => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:12:35.237277 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83286efc-26d3-4d43-af4e-09676284fe88 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:35.248027 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83286efc-26d3-4d43-af4e-09676284fe88 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:35.302217 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92912743-2931-4147-a977-245febfa45e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:35.302867 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92912743-2931-4147-a977-245febfa45e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:35.303148 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92912743-2931-4147-a977-245febfa45e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:35.303424 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92912743-2931-4147-a977-245febfa45e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:35.304171 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92912743-2931-4147-a977-245febfa45e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:35.305759 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92912743-2931-4147-a977-245febfa45e9 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:35.315782 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1286/2551] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:12:35 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:12:35.319774 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381e890e-79b4-4c17-a4f1-c02fcbbbf6ee admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:35.330093 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381e890e-79b4-4c17-a4f1-c02fcbbbf6ee admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:35.334601 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94b6f5a-f31d-47d2-a982-f3b1d98d0325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:35.335470 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94b6f5a-f31d-47d2-a982-f3b1d98d0325 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:35.335754 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94b6f5a-f31d-47d2-a982-f3b1d98d0325 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:35.336060 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94b6f5a-f31d-47d2-a982-f3b1d98d0325 admin admin] PATH_INFO: `/v3/projects/d29798d941fa4b168bee64e2b9f029a3/users/50a36e23d43d433ba261781b1d6de632/roles/b43d5a8247e244869c2c70f986f42e8a` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:35.338042 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94b6f5a-f31d-47d2-a982-f3b1d98d0325 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d29798d941fa4b168bee64e2b9f029a3, user_id=50a36e23d43d433ba261781b1d6de632, role_id=b43d5a8247e244869c2c70f986f42e8a)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:35.340728 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94b6f5a-f31d-47d2-a982-f3b1d98d0325 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:35.351282 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1266/2552] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:12:35 2022] PUT /identity/v3/projects/d29798d941fa4b168bee64e2b9f029a3/users/50a36e23d43d433ba261781b1d6de632/roles/b43d5a8247e244869c2c70f986f42e8a => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:12:35.362436 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b207de-c63b-43da-a4f8-8eebb5859873 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:35.362436 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b207de-c63b-43da-a4f8-8eebb5859873 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:35.362436 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b207de-c63b-43da-a4f8-8eebb5859873 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:35.395445 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-63b207de-c63b-43da-a4f8-8eebb5859873 None None] MFA Rules not processed for user `50a36e23d43d433ba261781b1d6de632`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:12:35.396662 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-63b207de-c63b-43da-a4f8-8eebb5859873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:12:35.464291 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1287/2553] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:12:35 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:12:35.475998 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94b6f5a-f31d-47d2-a982-f3b1d98d0325 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:35.477709 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c94b6f5a-f31d-47d2-a982-f3b1d98d0325 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:12:35.498715 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:12:35.537043 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94b6f5a-f31d-47d2-a982-f3b1d98d0325 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:35.540812 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe78f03f-54e8-4b86-aeec-16cc1873344f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6cc95a563b941ac92ab283aa0d5a5d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:35.541464 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe78f03f-54e8-4b86-aeec-16cc1873344f service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:35.541746 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe78f03f-54e8-4b86-aeec-16cc1873344f service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:35.542024 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe78f03f-54e8-4b86-aeec-16cc1873344f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:35.600010 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe78f03f-54e8-4b86-aeec-16cc1873344f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:35.601514 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe78f03f-54e8-4b86-aeec-16cc1873344f service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:35.611299 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1267/2554] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:12:35 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:12:37.213406 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b207de-c63b-43da-a4f8-8eebb5859873 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:37.215993 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-63b207de-c63b-43da-a4f8-8eebb5859873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:12:37.227118 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b207de-c63b-43da-a4f8-8eebb5859873 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:37.230624 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2587cea-8317-4bef-b2ce-402550bc2c9d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6cc95a563b941ac92ab283aa0d5a5d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:37.231939 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2587cea-8317-4bef-b2ce-402550bc2c9d service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:37.232220 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2587cea-8317-4bef-b2ce-402550bc2c9d service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:37.232492 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2587cea-8317-4bef-b2ce-402550bc2c9d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:37.241732 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2587cea-8317-4bef-b2ce-402550bc2c9d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:37.243123 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2587cea-8317-4bef-b2ce-402550bc2c9d service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:37.252551 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1288/2555] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:12:37 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:12:40.208066 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe78f03f-54e8-4b86-aeec-16cc1873344f service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:40.209459 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-fe78f03f-54e8-4b86-aeec-16cc1873344f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:12:40.266448 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe78f03f-54e8-4b86-aeec-16cc1873344f service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:40.270031 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e867a0-da4e-44a7-ab14-2bbe28c9f44e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:40.270668 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e867a0-da4e-44a7-ab14-2bbe28c9f44e service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:40.270954 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e867a0-da4e-44a7-ab14-2bbe28c9f44e service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:40.271233 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e867a0-da4e-44a7-ab14-2bbe28c9f44e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:40.273083 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d1e867a0-da4e-44a7-ab14-2bbe28c9f44e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:12:40.337943 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e867a0-da4e-44a7-ab14-2bbe28c9f44e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:40.339376 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e867a0-da4e-44a7-ab14-2bbe28c9f44e service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:40.350212 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1268/2556] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:12:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:12:41.415054 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2587cea-8317-4bef-b2ce-402550bc2c9d service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:41.425821 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2587cea-8317-4bef-b2ce-402550bc2c9d service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:41.430317 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdabeb2-154f-469e-b162-6d16325ed0ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:41.431173 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdabeb2-154f-469e-b162-6d16325ed0ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:41.431459 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdabeb2-154f-469e-b162-6d16325ed0ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:41.431741 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdabeb2-154f-469e-b162-6d16325ed0ad admin admin] PATH_INFO: `/v3/users/5b5d9e7ffa4749ee9494952940e582b5` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:41.433100 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdabeb2-154f-469e-b162-6d16325ed0ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b5d9e7ffa4749ee9494952940e582b5)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:41.435625 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdabeb2-154f-469e-b162-6d16325ed0ad admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:41.457909 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:12:41.601579 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4cdabeb2-154f-469e-b162-6d16325ed0ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b5d9e7ffa4749ee9494952940e582b5'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:41.609022 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4cdabeb2-154f-469e-b162-6d16325ed0ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b5d9e7ffa4749ee9494952940e582b5'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:41.656493 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4cdabeb2-154f-469e-b162-6d16325ed0ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b5d9e7ffa4749ee9494952940e582b5'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:41.657077 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4cdabeb2-154f-469e-b162-6d16325ed0ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b5d9e7ffa4749ee9494952940e582b5'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:41.670013 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4cdabeb2-154f-469e-b162-6d16325ed0ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b5d9e7ffa4749ee9494952940e582b5'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:41.676043 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1289/2557] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:12:41 2022] DELETE /identity/v3/users/5b5d9e7ffa4749ee9494952940e582b5 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:12:41.680800 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef03e8ce-5318-4be1-ba7c-a5f08b073d62 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:41.681296 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef03e8ce-5318-4be1-ba7c-a5f08b073d62 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:41.681497 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef03e8ce-5318-4be1-ba7c-a5f08b073d62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:41.698879 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-ef03e8ce-5318-4be1-ba7c-a5f08b073d62 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:12:41.700181 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-ef03e8ce-5318-4be1-ba7c-a5f08b073d62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:12:41.733471 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:12:41.777674 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1269/2558] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:12:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:12:41.789878 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdabeb2-154f-469e-b162-6d16325ed0ad admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:41.791293 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-4cdabeb2-154f-469e-b162-6d16325ed0ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:12:41.865577 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdabeb2-154f-469e-b162-6d16325ed0ad admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:41.871650 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a891dfe1-5f6d-4a22-9bfa-a341fdef3750 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:41.872423 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a891dfe1-5f6d-4a22-9bfa-a341fdef3750 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:41.872708 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a891dfe1-5f6d-4a22-9bfa-a341fdef3750 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:41.873656 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a891dfe1-5f6d-4a22-9bfa-a341fdef3750 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:41.885152 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a891dfe1-5f6d-4a22-9bfa-a341fdef3750 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:41.886869 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a891dfe1-5f6d-4a22-9bfa-a341fdef3750 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:41.899236 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1290/2559] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:12:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:12:42.270291 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef03e8ce-5318-4be1-ba7c-a5f08b073d62 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:42.271892 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-ef03e8ce-5318-4be1-ba7c-a5f08b073d62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:12:42.295636 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef03e8ce-5318-4be1-ba7c-a5f08b073d62 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:42.301116 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27da5f54-8670-4fec-991a-9b1f7cddcda8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:42.302138 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27da5f54-8670-4fec-991a-9b1f7cddcda8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:42.302541 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27da5f54-8670-4fec-991a-9b1f7cddcda8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:42.302916 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27da5f54-8670-4fec-991a-9b1f7cddcda8 admin admin] PATH_INFO: `/v3/projects/9f1fb2b9bc614686adfdd494db94558b` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:42.304331 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27da5f54-8670-4fec-991a-9b1f7cddcda8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f1fb2b9bc614686adfdd494db94558b)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:42.306611 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27da5f54-8670-4fec-991a-9b1f7cddcda8 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:42.429268 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-27da5f54-8670-4fec-991a-9b1f7cddcda8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f1fb2b9bc614686adfdd494db94558b', 'request_id': 'req-27da5f54-8670-4fec-991a-9b1f7cddcda8'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:42.435963 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-27da5f54-8670-4fec-991a-9b1f7cddcda8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f1fb2b9bc614686adfdd494db94558b', 'request_id': 'req-27da5f54-8670-4fec-991a-9b1f7cddcda8'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:42.442257 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-27da5f54-8670-4fec-991a-9b1f7cddcda8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f1fb2b9bc614686adfdd494db94558b', 'request_id': 'req-27da5f54-8670-4fec-991a-9b1f7cddcda8'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:42.530110 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-27da5f54-8670-4fec-991a-9b1f7cddcda8 admin admin] The token cache is being invalidate because project 9f1fb2b9bc614686adfdd494db94558b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:12:42.530451 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-27da5f54-8670-4fec-991a-9b1f7cddcda8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:42.531070 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-27da5f54-8670-4fec-991a-9b1f7cddcda8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:42.532652 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1270/2560] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:12:42 2022] DELETE /identity/v3/projects/9f1fb2b9bc614686adfdd494db94558b => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:12:42.536559 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a891dfe1-5f6d-4a22-9bfa-a341fdef3750 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:42.546780 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a891dfe1-5f6d-4a22-9bfa-a341fdef3750 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:42.608365 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a30fc5-65f2-4040-9562-23b52734c450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:42.609294 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a30fc5-65f2-4040-9562-23b52734c450 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:42.609576 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a30fc5-65f2-4040-9562-23b52734c450 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:42.609848 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a30fc5-65f2-4040-9562-23b52734c450 admin admin] PATH_INFO: `/v3/users/50a36e23d43d433ba261781b1d6de632` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:42.611016 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a30fc5-65f2-4040-9562-23b52734c450 admin admin] RBAC: Authorizing `identity:delete_user(user_id=50a36e23d43d433ba261781b1d6de632)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:42.612943 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a30fc5-65f2-4040-9562-23b52734c450 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:42.818844 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:12:42.822008 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a6a30fc5-65f2-4040-9562-23b52734c450 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50a36e23d43d433ba261781b1d6de632'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:42.831006 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a6a30fc5-65f2-4040-9562-23b52734c450 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50a36e23d43d433ba261781b1d6de632'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:42.890423 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a6a30fc5-65f2-4040-9562-23b52734c450 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50a36e23d43d433ba261781b1d6de632'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:42.891050 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a6a30fc5-65f2-4040-9562-23b52734c450 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50a36e23d43d433ba261781b1d6de632'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:42.904011 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-a6a30fc5-65f2-4040-9562-23b52734c450 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50a36e23d43d433ba261781b1d6de632'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:42.911163 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1291/2561] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:12:42 2022] DELETE /identity/v3/users/50a36e23d43d433ba261781b1d6de632 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:12:43.418758 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27da5f54-8670-4fec-991a-9b1f7cddcda8 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:43.420459 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-27da5f54-8670-4fec-991a-9b1f7cddcda8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:12:43.449386 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27da5f54-8670-4fec-991a-9b1f7cddcda8 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:43.541086 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7a6d59-34cc-4dbe-b21e-6505e4af4848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:43.541962 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7a6d59-34cc-4dbe-b21e-6505e4af4848 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:43.542244 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7a6d59-34cc-4dbe-b21e-6505e4af4848 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:43.542517 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7a6d59-34cc-4dbe-b21e-6505e4af4848 admin admin] PATH_INFO: `/v3/projects/d29798d941fa4b168bee64e2b9f029a3` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:43.543765 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7a6d59-34cc-4dbe-b21e-6505e4af4848 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d29798d941fa4b168bee64e2b9f029a3)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:43.545842 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7a6d59-34cc-4dbe-b21e-6505e4af4848 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:43.639992 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:12:43.641261 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2e7a6d59-34cc-4dbe-b21e-6505e4af4848 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd29798d941fa4b168bee64e2b9f029a3', 'request_id': 'req-2e7a6d59-34cc-4dbe-b21e-6505e4af4848'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:43.646859 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2e7a6d59-34cc-4dbe-b21e-6505e4af4848 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd29798d941fa4b168bee64e2b9f029a3', 'request_id': 'req-2e7a6d59-34cc-4dbe-b21e-6505e4af4848'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:43.654165 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2e7a6d59-34cc-4dbe-b21e-6505e4af4848 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd29798d941fa4b168bee64e2b9f029a3', 'request_id': 'req-2e7a6d59-34cc-4dbe-b21e-6505e4af4848'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:43.719731 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2e7a6d59-34cc-4dbe-b21e-6505e4af4848 admin admin] The token cache is being invalidate because project d29798d941fa4b168bee64e2b9f029a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:12:43.719971 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2e7a6d59-34cc-4dbe-b21e-6505e4af4848 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:43.720613 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-2e7a6d59-34cc-4dbe-b21e-6505e4af4848 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:43.722215 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1271/2562] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:12:43 2022] DELETE /identity/v3/projects/d29798d941fa4b168bee64e2b9f029a3 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:12:43.728001 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a938ad14-656f-43c4-8090-f5fc0c97778f None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:43.728602 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a938ad14-656f-43c4-8090-f5fc0c97778f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:43.728602 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a938ad14-656f-43c4-8090-f5fc0c97778f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:43.746728 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-a938ad14-656f-43c4-8090-f5fc0c97778f None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:12:43.748510 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a938ad14-656f-43c4-8090-f5fc0c97778f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:12:43.816418 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1292/2563] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:12:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 10 13:12:43.824273 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becd3603-7395-4e31-81b0-f51ced0caec2 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:43.824565 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becd3603-7395-4e31-81b0-f51ced0caec2 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:43.824848 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becd3603-7395-4e31-81b0-f51ced0caec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:43.844188 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-becd3603-7395-4e31-81b0-f51ced0caec2 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:12:43.845400 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-becd3603-7395-4e31-81b0-f51ced0caec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:12:43.859377 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1272/2564] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:12:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:12:43.865892 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a938ad14-656f-43c4-8090-f5fc0c97778f None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:43.876903 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a938ad14-656f-43c4-8090-f5fc0c97778f None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:43.881841 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b2e2af-3c99-45f6-83c3-58f379829feb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:43.883495 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b2e2af-3c99-45f6-83c3-58f379829feb admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:43.883495 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b2e2af-3c99-45f6-83c3-58f379829feb admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:43.883495 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b2e2af-3c99-45f6-83c3-58f379829feb admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:43.883780 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b2e2af-3c99-45f6-83c3-58f379829feb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 13:12:43.884094 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b2e2af-3c99-45f6-83c3-58f379829feb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:43.885881 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b2e2af-3c99-45f6-83c3-58f379829feb admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:43.898359 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1293/2565] 10.222.0.27 () {58 vars in 1152 bytes} [Fri Jun 10 13:12:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 13:12:43.902714 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becd3603-7395-4e31-81b0-f51ced0caec2 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:43.913024 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becd3603-7395-4e31-81b0-f51ced0caec2 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:43.917680 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44e18ab-1dca-4af2-a052-31d5c04271dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:43.918328 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44e18ab-1dca-4af2-a052-31d5c04271dc admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:43.918610 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44e18ab-1dca-4af2-a052-31d5c04271dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:43.918928 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44e18ab-1dca-4af2-a052-31d5c04271dc admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:43.920155 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44e18ab-1dca-4af2-a052-31d5c04271dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:43.922040 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44e18ab-1dca-4af2-a052-31d5c04271dc admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:44.016704 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1273/2566] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:12:43 2022] POST /identity/v3/projects => generated 367 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:12:44.017386 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b2e2af-3c99-45f6-83c3-58f379829feb admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:44.017633 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44e18ab-1dca-4af2-a052-31d5c04271dc admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:44.019338 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d1b2e2af-3c99-45f6-83c3-58f379829feb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:12:44.035729 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b2e2af-3c99-45f6-83c3-58f379829feb admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:44.095633 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591a6cd7-62b4-4926-b963-7995065014b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:44.096669 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591a6cd7-62b4-4926-b963-7995065014b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:44.096959 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591a6cd7-62b4-4926-b963-7995065014b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:44.097236 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591a6cd7-62b4-4926-b963-7995065014b3 admin admin] PATH_INFO: `/v3/users/8e32dbeb85bd4fb889d6dde932c60853` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:44.098427 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591a6cd7-62b4-4926-b963-7995065014b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e32dbeb85bd4fb889d6dde932c60853)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:44.100364 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591a6cd7-62b4-4926-b963-7995065014b3 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:44.107397 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44e18ab-1dca-4af2-a052-31d5c04271dc admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:44.111917 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72325913-a8d9-4c8e-a3bf-1c21097a9a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:44.112583 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72325913-a8d9-4c8e-a3bf-1c21097a9a17 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:44.112864 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72325913-a8d9-4c8e-a3bf-1c21097a9a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:44.113136 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72325913-a8d9-4c8e-a3bf-1c21097a9a17 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:44.113822 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72325913-a8d9-4c8e-a3bf-1c21097a9a17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:44.115710 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72325913-a8d9-4c8e-a3bf-1c21097a9a17 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:44.234230 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1274/2567] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:12:44 2022] POST /identity/v3/users => generated 284 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:12:44.239569 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72325913-a8d9-4c8e-a3bf-1c21097a9a17 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:44.249298 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72325913-a8d9-4c8e-a3bf-1c21097a9a17 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:44.253981 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2916f9-6a78-4d84-9a3c-71c1bbbb19ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:44.254621 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2916f9-6a78-4d84-9a3c-71c1bbbb19ba admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:44.254954 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2916f9-6a78-4d84-9a3c-71c1bbbb19ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:44.255174 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2916f9-6a78-4d84-9a3c-71c1bbbb19ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:44.256219 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2916f9-6a78-4d84-9a3c-71c1bbbb19ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:44.258193 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2916f9-6a78-4d84-9a3c-71c1bbbb19ba admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:44.265506 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-591a6cd7-62b4-4926-b963-7995065014b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e32dbeb85bd4fb889d6dde932c60853'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:44.267554 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:12:44.269370 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1275/2568] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:12:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 10 13:12:44.273253 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2916f9-6a78-4d84-9a3c-71c1bbbb19ba admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:44.273498 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-591a6cd7-62b4-4926-b963-7995065014b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e32dbeb85bd4fb889d6dde932c60853'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:44.283830 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2916f9-6a78-4d84-9a3c-71c1bbbb19ba admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:44.288557 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72cb174-ce57-4fce-a037-e7dffba4679a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:44.289224 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72cb174-ce57-4fce-a037-e7dffba4679a admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:44.289500 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72cb174-ce57-4fce-a037-e7dffba4679a admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:44.289783 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72cb174-ce57-4fce-a037-e7dffba4679a admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:44.290524 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72cb174-ce57-4fce-a037-e7dffba4679a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:44.292165 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72cb174-ce57-4fce-a037-e7dffba4679a admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:44.302534 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1276/2569] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:12:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 10 13:12:44.306335 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72cb174-ce57-4fce-a037-e7dffba4679a admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:44.315442 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72cb174-ce57-4fce-a037-e7dffba4679a admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:44.319113 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-591a6cd7-62b4-4926-b963-7995065014b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e32dbeb85bd4fb889d6dde932c60853'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:44.319828 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-591a6cd7-62b4-4926-b963-7995065014b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e32dbeb85bd4fb889d6dde932c60853'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:44.320076 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed50ecc-3c05-46f1-9d57-065a4bd68b40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:44.320919 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed50ecc-3c05-46f1-9d57-065a4bd68b40 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:44.321198 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed50ecc-3c05-46f1-9d57-065a4bd68b40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:44.321471 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed50ecc-3c05-46f1-9d57-065a4bd68b40 admin admin] PATH_INFO: `/v3/projects/a9769b685d9942439293d550ee5d1f3b/users/4fdd3e19e7884696ac1fbeb6ba793165/roles/43106c664e1b40de984c0c62701149c8` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:44.333422 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-591a6cd7-62b4-4926-b963-7995065014b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e32dbeb85bd4fb889d6dde932c60853'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:44.336394 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed50ecc-3c05-46f1-9d57-065a4bd68b40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9769b685d9942439293d550ee5d1f3b, user_id=4fdd3e19e7884696ac1fbeb6ba793165, role_id=43106c664e1b40de984c0c62701149c8)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:44.339051 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed50ecc-3c05-46f1-9d57-065a4bd68b40 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:44.339436 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1294/2570] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:12:44 2022] DELETE /identity/v3/users/8e32dbeb85bd4fb889d6dde932c60853 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 10 13:12:44.406224 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1277/2571] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:12:44 2022] PUT /identity/v3/projects/a9769b685d9942439293d550ee5d1f3b/users/4fdd3e19e7884696ac1fbeb6ba793165/roles/43106c664e1b40de984c0c62701149c8 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 10 13:12:44.415977 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f84b41-c5cb-4473-8e3d-1a8c5ce2d7ab None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:44.416272 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f84b41-c5cb-4473-8e3d-1a8c5ce2d7ab None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:44.416556 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f84b41-c5cb-4473-8e3d-1a8c5ce2d7ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:44.445406 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-f8f84b41-c5cb-4473-8e3d-1a8c5ce2d7ab None None] MFA Rules not processed for user `4fdd3e19e7884696ac1fbeb6ba793165`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:12:44.446677 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f8f84b41-c5cb-4473-8e3d-1a8c5ce2d7ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:12:44.526546 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1295/2572] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:12:44 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:12:44.531141 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed50ecc-3c05-46f1-9d57-065a4bd68b40 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:44.532918 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-8ed50ecc-3c05-46f1-9d57-065a4bd68b40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:12:44.572324 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f84b41-c5cb-4473-8e3d-1a8c5ce2d7ab None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:44.576092 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f8f84b41-c5cb-4473-8e3d-1a8c5ce2d7ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:12:44.587209 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=91,keystone:DELETE=3,keystone:INSERT=6 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:12:44.600612 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed50ecc-3c05-46f1-9d57-065a4bd68b40 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:44.606063 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c85b38-7875-4f7c-8c3c-01264dd144ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:44.606760 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c85b38-7875-4f7c-8c3c-01264dd144ac admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:44.607103 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c85b38-7875-4f7c-8c3c-01264dd144ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:44.607378 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c85b38-7875-4f7c-8c3c-01264dd144ac admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:44.608103 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c85b38-7875-4f7c-8c3c-01264dd144ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:44.609992 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c85b38-7875-4f7c-8c3c-01264dd144ac admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:44.648292 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f84b41-c5cb-4473-8e3d-1a8c5ce2d7ab None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:44.652832 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb94be5a-8195-4c68-97c2-5e7b64b7c837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:44.653724 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb94be5a-8195-4c68-97c2-5e7b64b7c837 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:44.654001 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb94be5a-8195-4c68-97c2-5e7b64b7c837 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:44.654345 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb94be5a-8195-4c68-97c2-5e7b64b7c837 admin admin] PATH_INFO: `/v3/projects/d06694ad116841d49f4bb980042ca4c3` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:44.655768 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb94be5a-8195-4c68-97c2-5e7b64b7c837 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d06694ad116841d49f4bb980042ca4c3)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:44.657679 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb94be5a-8195-4c68-97c2-5e7b64b7c837 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:44.700903 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1278/2573] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:12:44 2022] POST /identity/v3/projects => generated 367 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 10 13:12:44.702081 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c85b38-7875-4f7c-8c3c-01264dd144ac admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:44.760980 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-fb94be5a-8195-4c68-97c2-5e7b64b7c837 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd06694ad116841d49f4bb980042ca4c3', 'request_id': 'req-fb94be5a-8195-4c68-97c2-5e7b64b7c837'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:44.761843 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c85b38-7875-4f7c-8c3c-01264dd144ac admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:44.767121 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-fb94be5a-8195-4c68-97c2-5e7b64b7c837 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd06694ad116841d49f4bb980042ca4c3', 'request_id': 'req-fb94be5a-8195-4c68-97c2-5e7b64b7c837'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:44.768253 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=90,keystone:INSERT=1,keystone:DELETE=9 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:12:44.777582 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-fb94be5a-8195-4c68-97c2-5e7b64b7c837 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd06694ad116841d49f4bb980042ca4c3', 'request_id': 'req-fb94be5a-8195-4c68-97c2-5e7b64b7c837'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:44.817950 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43863c44-3935-4c76-8c09-218bcf5c8723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:44.818729 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43863c44-3935-4c76-8c09-218bcf5c8723 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:44.819016 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43863c44-3935-4c76-8c09-218bcf5c8723 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:44.819294 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43863c44-3935-4c76-8c09-218bcf5c8723 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:44.820002 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43863c44-3935-4c76-8c09-218bcf5c8723 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:44.821944 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43863c44-3935-4c76-8c09-218bcf5c8723 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:44.844048 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-fb94be5a-8195-4c68-97c2-5e7b64b7c837 admin admin] The token cache is being invalidate because project d06694ad116841d49f4bb980042ca4c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:12:44.844337 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-fb94be5a-8195-4c68-97c2-5e7b64b7c837 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:44.844966 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-fb94be5a-8195-4c68-97c2-5e7b64b7c837 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:44.846507 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1296/2574] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:12:44 2022] DELETE /identity/v3/projects/d06694ad116841d49f4bb980042ca4c3 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 10 13:12:44.851427 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88bebda-e9aa-466f-bffb-e4ceebeb8ea7 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:44.851992 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88bebda-e9aa-466f-bffb-e4ceebeb8ea7 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:44.852280 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88bebda-e9aa-466f-bffb-e4ceebeb8ea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:44.870918 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-b88bebda-e9aa-466f-bffb-e4ceebeb8ea7 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:12:44.872759 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b88bebda-e9aa-466f-bffb-e4ceebeb8ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:12:44.880833 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1297/2575] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:12:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 10 13:12:44.888893 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bffb7fa-8a5e-4718-9870-04a6f198b24b None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:44.889181 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bffb7fa-8a5e-4718-9870-04a6f198b24b None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:44.889457 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bffb7fa-8a5e-4718-9870-04a6f198b24b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:44.907197 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-9bffb7fa-8a5e-4718-9870-04a6f198b24b None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:12:44.908429 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9bffb7fa-8a5e-4718-9870-04a6f198b24b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:12:44.916503 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1298/2576] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:12:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 10 13:12:44.920958 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1279/2577] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:12:44 2022] POST /identity/v3/users => generated 290 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 10 13:12:44.922419 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bffb7fa-8a5e-4718-9870-04a6f198b24b None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:44.926929 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43863c44-3935-4c76-8c09-218bcf5c8723 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:44.929301 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-43863c44-3935-4c76-8c09-218bcf5c8723 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:12:44.936154 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bffb7fa-8a5e-4718-9870-04a6f198b24b None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:44.941341 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf94da0-aefe-43e7-a86b-1b62658a7bb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:44.941924 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43863c44-3935-4c76-8c09-218bcf5c8723 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:44.942110 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf94da0-aefe-43e7-a86b-1b62658a7bb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:44.942278 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf94da0-aefe-43e7-a86b-1b62658a7bb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:44.942564 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf94da0-aefe-43e7-a86b-1b62658a7bb0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:44.943438 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf94da0-aefe-43e7-a86b-1b62658a7bb0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 13:12:44.943748 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf94da0-aefe-43e7-a86b-1b62658a7bb0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:44.945908 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf94da0-aefe-43e7-a86b-1b62658a7bb0 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:44.946799 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ddd92c-1f71-4894-95e1-d75f8b358e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:44.947458 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ddd92c-1f71-4894-95e1-d75f8b358e8b admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:44.947740 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ddd92c-1f71-4894-95e1-d75f8b358e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:44.948048 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ddd92c-1f71-4894-95e1-d75f8b358e8b admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:44.948807 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ddd92c-1f71-4894-95e1-d75f8b358e8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:44.950382 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ddd92c-1f71-4894-95e1-d75f8b358e8b admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:44.957993 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1299/2578] 10.222.0.27 () {58 vars in 1152 bytes} [Fri Jun 10 13:12:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 13:12:44.961072 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1280/2579] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:12:44 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 10 13:12:44.962300 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf94da0-aefe-43e7-a86b-1b62658a7bb0 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:44.964787 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ddd92c-1f71-4894-95e1-d75f8b358e8b admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:44.972173 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf94da0-aefe-43e7-a86b-1b62658a7bb0 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:44.977282 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ddd92c-1f71-4894-95e1-d75f8b358e8b admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:44.977397 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeb0ac0-2d73-42de-82de-0c73eeb97292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:44.977544 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeb0ac0-2d73-42de-82de-0c73eeb97292 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:44.977831 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeb0ac0-2d73-42de-82de-0c73eeb97292 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:44.978124 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeb0ac0-2d73-42de-82de-0c73eeb97292 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:44.978784 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abeb0ac0-2d73-42de-82de-0c73eeb97292 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:44.979740 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32f7bee-467d-4e79-ab82-6d9c43e10a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:44.980654 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32f7bee-467d-4e79-ab82-6d9c43e10a54 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:44.980796 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abeb0ac0-2d73-42de-82de-0c73eeb97292 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:44.980958 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32f7bee-467d-4e79-ab82-6d9c43e10a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:44.981210 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32f7bee-467d-4e79-ab82-6d9c43e10a54 admin admin] PATH_INFO: `/v3/projects/e354352fea384370bb5a89fed919ab9d/users/69a7f8b327fa4fcdabbdf7f7c19b219a/roles/b43d5a8247e244869c2c70f986f42e8a` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:44.998513 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32f7bee-467d-4e79-ab82-6d9c43e10a54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e354352fea384370bb5a89fed919ab9d, user_id=69a7f8b327fa4fcdabbdf7f7c19b219a, role_id=b43d5a8247e244869c2c70f986f42e8a)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:45.001187 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32f7bee-467d-4e79-ab82-6d9c43e10a54 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:45.053886 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1300/2580] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:12:44 2022] POST /identity/v3/projects => generated 375 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 10 13:12:45.056001 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeb0ac0-2d73-42de-82de-0c73eeb97292 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:45.063248 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1281/2581] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:12:44 2022] PUT /identity/v3/projects/e354352fea384370bb5a89fed919ab9d/users/69a7f8b327fa4fcdabbdf7f7c19b219a/roles/b43d5a8247e244869c2c70f986f42e8a => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 10 13:12:45.067202 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32f7bee-467d-4e79-ab82-6d9c43e10a54 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:45.109444 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:12:45.118726 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeb0ac0-2d73-42de-82de-0c73eeb97292 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:45.130874 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32f7bee-467d-4e79-ab82-6d9c43e10a54 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:45.135669 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe3589c-2bd1-4d24-a59f-a922634e3cb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:45.136374 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe3589c-2bd1-4d24-a59f-a922634e3cb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:45.136680 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe3589c-2bd1-4d24-a59f-a922634e3cb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:45.136964 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe3589c-2bd1-4d24-a59f-a922634e3cb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:45.137751 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe3589c-2bd1-4d24-a59f-a922634e3cb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:45.139350 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe3589c-2bd1-4d24-a59f-a922634e3cb8 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:45.152496 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1282/2582] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:12:45 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 10 13:12:45.153812 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe3589c-2bd1-4d24-a59f-a922634e3cb8 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:45.163776 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe3589c-2bd1-4d24-a59f-a922634e3cb8 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:45.168551 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f961f2-73fd-4ba3-b7c0-900795546398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:45.169956 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f961f2-73fd-4ba3-b7c0-900795546398 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:45.170239 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f961f2-73fd-4ba3-b7c0-900795546398 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:45.170515 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f961f2-73fd-4ba3-b7c0-900795546398 admin admin] PATH_INFO: `/v3/projects/e354352fea384370bb5a89fed919ab9d/users/69a7f8b327fa4fcdabbdf7f7c19b219a/roles/b43d5a8247e244869c2c70f986f42e8a` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:45.172515 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f961f2-73fd-4ba3-b7c0-900795546398 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e354352fea384370bb5a89fed919ab9d, user_id=69a7f8b327fa4fcdabbdf7f7c19b219a, role_id=b43d5a8247e244869c2c70f986f42e8a)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:45.175121 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f961f2-73fd-4ba3-b7c0-900795546398 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:45.175975 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4584f426-87a7-4221-8830-7e9766b4d6be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:45.176650 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4584f426-87a7-4221-8830-7e9766b4d6be admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:45.176937 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4584f426-87a7-4221-8830-7e9766b4d6be admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:45.177217 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4584f426-87a7-4221-8830-7e9766b4d6be admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:45.177895 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4584f426-87a7-4221-8830-7e9766b4d6be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:45.180386 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4584f426-87a7-4221-8830-7e9766b4d6be admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:45.187252 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1283/2583] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:12:45 2022] PUT /identity/v3/projects/e354352fea384370bb5a89fed919ab9d/users/69a7f8b327fa4fcdabbdf7f7c19b219a/roles/b43d5a8247e244869c2c70f986f42e8a => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 10 13:12:45.197527 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fc6e45-f804-43f0-93d5-57c21e199eee None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:45.197831 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fc6e45-f804-43f0-93d5-57c21e199eee None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:45.198134 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fc6e45-f804-43f0-93d5-57c21e199eee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:45.224446 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-56fc6e45-f804-43f0-93d5-57c21e199eee None None] MFA Rules not processed for user `69a7f8b327fa4fcdabbdf7f7c19b219a`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:12:45.225652 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-56fc6e45-f804-43f0-93d5-57c21e199eee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:12:45.280940 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1301/2584] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:12:45 2022] POST /identity/v3/users => generated 288 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 10 13:12:45.284618 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4584f426-87a7-4221-8830-7e9766b4d6be admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:45.293512 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1284/2585] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:12:45 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 10 13:12:45.294971 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4584f426-87a7-4221-8830-7e9766b4d6be admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:45.305344 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fc6e45-f804-43f0-93d5-57c21e199eee None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:45.306639 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-56fc6e45-f804-43f0-93d5-57c21e199eee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:12:45.327734 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:12:45.352021 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302f4dda-7678-476c-a22d-950c5baadce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:45.353205 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302f4dda-7678-476c-a22d-950c5baadce9 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:45.353890 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302f4dda-7678-476c-a22d-950c5baadce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:45.354421 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302f4dda-7678-476c-a22d-950c5baadce9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:45.355615 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302f4dda-7678-476c-a22d-950c5baadce9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:45.358029 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302f4dda-7678-476c-a22d-950c5baadce9 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:45.363328 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fc6e45-f804-43f0-93d5-57c21e199eee None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:45.366992 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34c59b5-6fae-4657-9683-eb942c05351b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6cc95a563b941ac92ab283aa0d5a5d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:45.367627 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34c59b5-6fae-4657-9683-eb942c05351b service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:45.367929 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34c59b5-6fae-4657-9683-eb942c05351b service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:45.368204 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34c59b5-6fae-4657-9683-eb942c05351b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:45.368621 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1302/2586] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:12:45 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 10 13:12:45.372572 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302f4dda-7678-476c-a22d-950c5baadce9 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:45.377590 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34c59b5-6fae-4657-9683-eb942c05351b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:45.379053 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34c59b5-6fae-4657-9683-eb942c05351b service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:45.383028 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302f4dda-7678-476c-a22d-950c5baadce9 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:45.387718 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c96516b-d88b-484e-91ca-2ddea03631fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:45.388977 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c96516b-d88b-484e-91ca-2ddea03631fe admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:45.389109 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1285/2587] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:12:45 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 10 13:12:45.389210 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c96516b-d88b-484e-91ca-2ddea03631fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:45.389210 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c96516b-d88b-484e-91ca-2ddea03631fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:45.389902 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c96516b-d88b-484e-91ca-2ddea03631fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:45.391488 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c96516b-d88b-484e-91ca-2ddea03631fe admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:45.402967 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1303/2588] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:12:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:12:45.407404 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34c59b5-6fae-4657-9683-eb942c05351b service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:45.417988 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34c59b5-6fae-4657-9683-eb942c05351b service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:45.423289 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feece616-b5be-4d2e-9bde-81e3e4567295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:45.424194 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feece616-b5be-4d2e-9bde-81e3e4567295 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:45.424480 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feece616-b5be-4d2e-9bde-81e3e4567295 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:45.424755 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feece616-b5be-4d2e-9bde-81e3e4567295 admin admin] PATH_INFO: `/v3/projects/41980b9b16534431ab38e4f376ea54f3/users/73e9cffb8fb441c181a2e23dafd3063f/roles/43106c664e1b40de984c0c62701149c8` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:45.439438 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feece616-b5be-4d2e-9bde-81e3e4567295 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41980b9b16534431ab38e4f376ea54f3, user_id=73e9cffb8fb441c181a2e23dafd3063f, role_id=43106c664e1b40de984c0c62701149c8)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:45.442069 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feece616-b5be-4d2e-9bde-81e3e4567295 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:45.491324 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1286/2589] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:12:45 2022] PUT /identity/v3/projects/41980b9b16534431ab38e4f376ea54f3/users/73e9cffb8fb441c181a2e23dafd3063f/roles/43106c664e1b40de984c0c62701149c8 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:12:45.503763 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033cefd3-08ac-4ff4-b7c8-a5fe314ed574 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:45.504165 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033cefd3-08ac-4ff4-b7c8-a5fe314ed574 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:45.504467 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033cefd3-08ac-4ff4-b7c8-a5fe314ed574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:45.530756 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-033cefd3-08ac-4ff4-b7c8-a5fe314ed574 None None] MFA Rules not processed for user `73e9cffb8fb441c181a2e23dafd3063f`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:12:45.532020 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-033cefd3-08ac-4ff4-b7c8-a5fe314ed574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:12:45.573433 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feece616-b5be-4d2e-9bde-81e3e4567295 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:45.598682 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1304/2590] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:12:45 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:12:45.600886 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:12:45.610626 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033cefd3-08ac-4ff4-b7c8-a5fe314ed574 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:45.612050 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-033cefd3-08ac-4ff4-b7c8-a5fe314ed574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:12:45.629875 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feece616-b5be-4d2e-9bde-81e3e4567295 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:45.633834 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82607ed8-b826-494b-8643-bb9fa5181d41 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6cc95a563b941ac92ab283aa0d5a5d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:45.634470 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82607ed8-b826-494b-8643-bb9fa5181d41 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:45.634757 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82607ed8-b826-494b-8643-bb9fa5181d41 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:45.635034 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82607ed8-b826-494b-8643-bb9fa5181d41 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:45.636849 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-82607ed8-b826-494b-8643-bb9fa5181d41 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:12:45.685916 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033cefd3-08ac-4ff4-b7c8-a5fe314ed574 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:45.691427 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612ef33b-14ba-4b1d-9e61-4483002fc50c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bd6caa461e84100a3d5fe5af5d80f3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:45.692101 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612ef33b-14ba-4b1d-9e61-4483002fc50c service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:45.692389 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612ef33b-14ba-4b1d-9e61-4483002fc50c service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:45.692673 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612ef33b-14ba-4b1d-9e61-4483002fc50c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:45.696220 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82607ed8-b826-494b-8643-bb9fa5181d41 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:45.697671 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82607ed8-b826-494b-8643-bb9fa5181d41 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:45.700804 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612ef33b-14ba-4b1d-9e61-4483002fc50c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:45.702391 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612ef33b-14ba-4b1d-9e61-4483002fc50c service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:45.706741 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1287/2591] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:12:45 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:12:45.711587 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1305/2592] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:12:45 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 10 13:12:45.927256 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82607ed8-b826-494b-8643-bb9fa5181d41 service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:45.990133 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82607ed8-b826-494b-8643-bb9fa5181d41 service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:45.994675 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdbfff2-05d0-4e56-959b-baed68bc07f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:45.995541 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdbfff2-05d0-4e56-959b-baed68bc07f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:45.995824 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdbfff2-05d0-4e56-959b-baed68bc07f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:46.000675 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdbfff2-05d0-4e56-959b-baed68bc07f6 admin admin] PATH_INFO: `/v3/users/73e9cffb8fb441c181a2e23dafd3063f` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:46.000675 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdbfff2-05d0-4e56-959b-baed68bc07f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=73e9cffb8fb441c181a2e23dafd3063f)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:46.000675 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdbfff2-05d0-4e56-959b-baed68bc07f6 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:46.126097 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-3fdbfff2-05d0-4e56-959b-baed68bc07f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73e9cffb8fb441c181a2e23dafd3063f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:46.133005 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-3fdbfff2-05d0-4e56-959b-baed68bc07f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73e9cffb8fb441c181a2e23dafd3063f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:46.233665 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-3fdbfff2-05d0-4e56-959b-baed68bc07f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73e9cffb8fb441c181a2e23dafd3063f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:46.234285 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-3fdbfff2-05d0-4e56-959b-baed68bc07f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73e9cffb8fb441c181a2e23dafd3063f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:46.246730 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-3fdbfff2-05d0-4e56-959b-baed68bc07f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73e9cffb8fb441c181a2e23dafd3063f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:46.248187 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:12:46.253102 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1288/2593] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:12:45 2022] DELETE /identity/v3/users/73e9cffb8fb441c181a2e23dafd3063f => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:12:46.258157 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f96475-1f09-4355-83b1-55835d89ea02 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:46.263965 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f96475-1f09-4355-83b1-55835d89ea02 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:46.263965 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f96475-1f09-4355-83b1-55835d89ea02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:46.277991 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-10f96475-1f09-4355-83b1-55835d89ea02 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:12:46.279891 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-10f96475-1f09-4355-83b1-55835d89ea02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:12:46.311491 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:12:46.340476 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1306/2594] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:12:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:12:46.352830 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdbfff2-05d0-4e56-959b-baed68bc07f6 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:46.354540 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-3fdbfff2-05d0-4e56-959b-baed68bc07f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:12:46.413270 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdbfff2-05d0-4e56-959b-baed68bc07f6 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:46.416946 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4084acfc-9c68-4c02-955a-bc78eb4d6088 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:46.417580 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4084acfc-9c68-4c02-955a-bc78eb4d6088 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:46.417863 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4084acfc-9c68-4c02-955a-bc78eb4d6088 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:46.418142 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4084acfc-9c68-4c02-955a-bc78eb4d6088 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:46.427006 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4084acfc-9c68-4c02-955a-bc78eb4d6088 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:46.428479 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4084acfc-9c68-4c02-955a-bc78eb4d6088 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:46.438347 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1289/2595] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:12:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:12:46.834765 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f96475-1f09-4355-83b1-55835d89ea02 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:46.836220 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-10f96475-1f09-4355-83b1-55835d89ea02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:12:46.850322 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f96475-1f09-4355-83b1-55835d89ea02 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:46.855001 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066e2253-8802-4d45-8715-8d9ecd85ae52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:46.855887 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066e2253-8802-4d45-8715-8d9ecd85ae52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:46.856189 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066e2253-8802-4d45-8715-8d9ecd85ae52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:46.856469 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066e2253-8802-4d45-8715-8d9ecd85ae52 admin admin] PATH_INFO: `/v3/projects/41980b9b16534431ab38e4f376ea54f3` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:46.857667 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066e2253-8802-4d45-8715-8d9ecd85ae52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=41980b9b16534431ab38e4f376ea54f3)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:46.860682 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066e2253-8802-4d45-8715-8d9ecd85ae52 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:46.958728 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-066e2253-8802-4d45-8715-8d9ecd85ae52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41980b9b16534431ab38e4f376ea54f3', 'request_id': 'req-066e2253-8802-4d45-8715-8d9ecd85ae52'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:46.964171 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-066e2253-8802-4d45-8715-8d9ecd85ae52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41980b9b16534431ab38e4f376ea54f3', 'request_id': 'req-066e2253-8802-4d45-8715-8d9ecd85ae52'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:46.970122 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-066e2253-8802-4d45-8715-8d9ecd85ae52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41980b9b16534431ab38e4f376ea54f3', 'request_id': 'req-066e2253-8802-4d45-8715-8d9ecd85ae52'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:47.020811 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-066e2253-8802-4d45-8715-8d9ecd85ae52 admin admin] The token cache is being invalidate because project 41980b9b16534431ab38e4f376ea54f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:12:47.021101 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-066e2253-8802-4d45-8715-8d9ecd85ae52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:47.021704 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-066e2253-8802-4d45-8715-8d9ecd85ae52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:47.023233 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1307/2596] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:12:46 2022] DELETE /identity/v3/projects/41980b9b16534431ab38e4f376ea54f3 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:12:47.028471 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74ee106-d7a3-44a2-8ad7-b7b3af7d8c01 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:47.028967 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74ee106-d7a3-44a2-8ad7-b7b3af7d8c01 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:47.029064 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74ee106-d7a3-44a2-8ad7-b7b3af7d8c01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:47.046971 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-b74ee106-d7a3-44a2-8ad7-b7b3af7d8c01 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:12:47.048190 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b74ee106-d7a3-44a2-8ad7-b7b3af7d8c01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:12:47.107514 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1290/2597] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:12:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:12:47.115349 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0108eae7-776e-4b39-a381-e8040f4f0b93 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:47.115636 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0108eae7-776e-4b39-a381-e8040f4f0b93 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:47.115943 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0108eae7-776e-4b39-a381-e8040f4f0b93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:47.134531 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-0108eae7-776e-4b39-a381-e8040f4f0b93 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:12:47.135720 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0108eae7-776e-4b39-a381-e8040f4f0b93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:12:47.143588 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1308/2598] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:12:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:12:47.147769 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74ee106-d7a3-44a2-8ad7-b7b3af7d8c01 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:47.157766 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74ee106-d7a3-44a2-8ad7-b7b3af7d8c01 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:47.162237 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d3997f-291f-448c-9d88-4582e87056f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:47.162892 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d3997f-291f-448c-9d88-4582e87056f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:47.163236 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d3997f-291f-448c-9d88-4582e87056f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:47.163464 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d3997f-291f-448c-9d88-4582e87056f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:47.164255 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d3997f-291f-448c-9d88-4582e87056f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 13:12:47.164567 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d3997f-291f-448c-9d88-4582e87056f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:47.166160 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d3997f-291f-448c-9d88-4582e87056f5 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:47.178249 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1291/2599] 10.222.0.27 () {58 vars in 1152 bytes} [Fri Jun 10 13:12:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 13:12:47.182442 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0108eae7-776e-4b39-a381-e8040f4f0b93 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:47.192342 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0108eae7-776e-4b39-a381-e8040f4f0b93 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:47.197033 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31280233-7fc6-4d08-b19e-19d2403ea09f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:47.197714 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31280233-7fc6-4d08-b19e-19d2403ea09f admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:47.198000 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31280233-7fc6-4d08-b19e-19d2403ea09f admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:47.198278 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31280233-7fc6-4d08-b19e-19d2403ea09f admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:47.198956 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31280233-7fc6-4d08-b19e-19d2403ea09f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:47.200880 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31280233-7fc6-4d08-b19e-19d2403ea09f admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:47.291604 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1309/2600] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:12:47 2022] POST /identity/v3/projects => generated 405 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:12:47.296053 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d3997f-291f-448c-9d88-4582e87056f5 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:47.306130 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d3997f-291f-448c-9d88-4582e87056f5 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:47.364975 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db2bf72-ef37-4bd1-9d84-146d812ae270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:47.365670 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db2bf72-ef37-4bd1-9d84-146d812ae270 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:47.371814 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db2bf72-ef37-4bd1-9d84-146d812ae270 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:47.372697 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db2bf72-ef37-4bd1-9d84-146d812ae270 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:47.373430 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db2bf72-ef37-4bd1-9d84-146d812ae270 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:47.375375 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db2bf72-ef37-4bd1-9d84-146d812ae270 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:47.461851 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1292/2601] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:12:47 2022] POST /identity/v3/users => generated 303 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 10 13:12:47.466731 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31280233-7fc6-4d08-b19e-19d2403ea09f admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:47.476883 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31280233-7fc6-4d08-b19e-19d2403ea09f admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:47.482098 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdadf8b-8419-4c84-b1cb-f72d9cba2f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:47.482752 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdadf8b-8419-4c84-b1cb-f72d9cba2f6c admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:47.483036 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdadf8b-8419-4c84-b1cb-f72d9cba2f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:47.483386 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdadf8b-8419-4c84-b1cb-f72d9cba2f6c admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:47.484147 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdadf8b-8419-4c84-b1cb-f72d9cba2f6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:47.485715 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdadf8b-8419-4c84-b1cb-f72d9cba2f6c admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:47.497179 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1310/2602] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:12:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:12:47.501062 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db2bf72-ef37-4bd1-9d84-146d812ae270 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:47.511126 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db2bf72-ef37-4bd1-9d84-146d812ae270 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:47.515728 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f58369-bcba-428b-bb64-3972f5729d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:47.516408 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f58369-bcba-428b-bb64-3972f5729d58 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:47.516695 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f58369-bcba-428b-bb64-3972f5729d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:47.516972 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f58369-bcba-428b-bb64-3972f5729d58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:47.517690 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f58369-bcba-428b-bb64-3972f5729d58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:47.519257 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f58369-bcba-428b-bb64-3972f5729d58 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:47.529979 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1293/2603] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:12:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:12:47.534197 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdadf8b-8419-4c84-b1cb-f72d9cba2f6c admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:47.545070 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdadf8b-8419-4c84-b1cb-f72d9cba2f6c admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:47.549703 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d046f175-9e6a-48c1-b14d-235a4f2e2e98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:47.550571 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d046f175-9e6a-48c1-b14d-235a4f2e2e98 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:47.550855 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d046f175-9e6a-48c1-b14d-235a4f2e2e98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:47.551132 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d046f175-9e6a-48c1-b14d-235a4f2e2e98 admin admin] PATH_INFO: `/v3/projects/f5b7f29801c546abb7a89068edf12cd7/users/6fe75f5f7c3d4aa2b747b2d3b1eb5348/roles/43106c664e1b40de984c0c62701149c8` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:47.565816 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d046f175-9e6a-48c1-b14d-235a4f2e2e98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5b7f29801c546abb7a89068edf12cd7, user_id=6fe75f5f7c3d4aa2b747b2d3b1eb5348, role_id=43106c664e1b40de984c0c62701149c8)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:47.568523 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d046f175-9e6a-48c1-b14d-235a4f2e2e98 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:47.638702 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1311/2604] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:12:47 2022] PUT /identity/v3/projects/f5b7f29801c546abb7a89068edf12cd7/users/6fe75f5f7c3d4aa2b747b2d3b1eb5348/roles/43106c664e1b40de984c0c62701149c8 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:12:47.649960 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f9ff7c-7189-4621-b41d-929cd30e7601 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:47.650252 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f9ff7c-7189-4621-b41d-929cd30e7601 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:47.650528 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f9ff7c-7189-4621-b41d-929cd30e7601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:47.664150 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:12:47.678675 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-a8f9ff7c-7189-4621-b41d-929cd30e7601 None None] MFA Rules not processed for user `6fe75f5f7c3d4aa2b747b2d3b1eb5348`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:12:47.679934 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a8f9ff7c-7189-4621-b41d-929cd30e7601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:12:47.743764 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1294/2605] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:12:47 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:12:47.748291 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d046f175-9e6a-48c1-b14d-235a4f2e2e98 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:47.807824 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:12:47.816184 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d046f175-9e6a-48c1-b14d-235a4f2e2e98 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:47.821452 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14db480d-38d7-4671-b64c-9e6864f2bfc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:47.822051 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14db480d-38d7-4671-b64c-9e6864f2bfc3 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:47.822391 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14db480d-38d7-4671-b64c-9e6864f2bfc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:47.822598 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14db480d-38d7-4671-b64c-9e6864f2bfc3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:47.823293 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14db480d-38d7-4671-b64c-9e6864f2bfc3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:47.826484 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14db480d-38d7-4671-b64c-9e6864f2bfc3 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:47.926225 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1312/2606] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:12:47 2022] POST /identity/v3/projects => generated 405 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:12:47.930254 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f9ff7c-7189-4621-b41d-929cd30e7601 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:47.941271 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f9ff7c-7189-4621-b41d-929cd30e7601 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:47.998112 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20108a33-8fae-4896-b2de-3a8ef2d194c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:47.998819 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20108a33-8fae-4896-b2de-3a8ef2d194c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:47.999108 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20108a33-8fae-4896-b2de-3a8ef2d194c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:47.999393 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20108a33-8fae-4896-b2de-3a8ef2d194c5 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:48.000229 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20108a33-8fae-4896-b2de-3a8ef2d194c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:48.002139 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20108a33-8fae-4896-b2de-3a8ef2d194c5 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:48.091793 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1295/2607] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:12:47 2022] POST /identity/v3/users => generated 309 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:12:48.095784 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14db480d-38d7-4671-b64c-9e6864f2bfc3 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:48.106279 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14db480d-38d7-4671-b64c-9e6864f2bfc3 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:48.111814 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d359f41-c511-4295-93ec-56ea9342d27e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:48.111814 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d359f41-c511-4295-93ec-56ea9342d27e admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:48.112095 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d359f41-c511-4295-93ec-56ea9342d27e admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:48.112195 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d359f41-c511-4295-93ec-56ea9342d27e admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:48.112849 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d359f41-c511-4295-93ec-56ea9342d27e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:48.114427 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d359f41-c511-4295-93ec-56ea9342d27e admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:48.124878 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1313/2608] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:12:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:12:48.128864 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20108a33-8fae-4896-b2de-3a8ef2d194c5 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:48.139339 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20108a33-8fae-4896-b2de-3a8ef2d194c5 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:48.144133 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349784df-3060-46c8-8d0c-5a8520a93320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:48.159822 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349784df-3060-46c8-8d0c-5a8520a93320 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:48.159822 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349784df-3060-46c8-8d0c-5a8520a93320 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:48.159822 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349784df-3060-46c8-8d0c-5a8520a93320 admin admin] PATH_INFO: `/v3/projects/0d22f3c79d3b4e7e8ce7d102842ea811/users/e1e659e4416044258f1d242ef86b98d0/roles/b43d5a8247e244869c2c70f986f42e8a` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:48.161312 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349784df-3060-46c8-8d0c-5a8520a93320 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d22f3c79d3b4e7e8ce7d102842ea811, user_id=e1e659e4416044258f1d242ef86b98d0, role_id=b43d5a8247e244869c2c70f986f42e8a)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:48.165276 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349784df-3060-46c8-8d0c-5a8520a93320 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:48.232977 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1296/2609] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:12:48 2022] PUT /identity/v3/projects/0d22f3c79d3b4e7e8ce7d102842ea811/users/e1e659e4416044258f1d242ef86b98d0/roles/b43d5a8247e244869c2c70f986f42e8a => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:12:48.236908 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d359f41-c511-4295-93ec-56ea9342d27e admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:48.246856 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d359f41-c511-4295-93ec-56ea9342d27e admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:48.302709 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77f4536-9c73-49e6-8a9e-907f494e3742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:48.303371 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77f4536-9c73-49e6-8a9e-907f494e3742 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:48.303655 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77f4536-9c73-49e6-8a9e-907f494e3742 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:48.303958 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77f4536-9c73-49e6-8a9e-907f494e3742 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:48.304674 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77f4536-9c73-49e6-8a9e-907f494e3742 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:48.306249 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77f4536-9c73-49e6-8a9e-907f494e3742 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:48.316442 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1314/2610] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:12:48 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 10 13:12:48.320533 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349784df-3060-46c8-8d0c-5a8520a93320 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:48.331320 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349784df-3060-46c8-8d0c-5a8520a93320 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:48.335926 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75afe151-8c90-46af-8f74-8615418e52dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:48.336800 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75afe151-8c90-46af-8f74-8615418e52dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:48.337082 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75afe151-8c90-46af-8f74-8615418e52dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:48.337361 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75afe151-8c90-46af-8f74-8615418e52dd admin admin] PATH_INFO: `/v3/projects/0d22f3c79d3b4e7e8ce7d102842ea811/users/e1e659e4416044258f1d242ef86b98d0/roles/b43d5a8247e244869c2c70f986f42e8a` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:48.339300 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75afe151-8c90-46af-8f74-8615418e52dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d22f3c79d3b4e7e8ce7d102842ea811, user_id=e1e659e4416044258f1d242ef86b98d0, role_id=b43d5a8247e244869c2c70f986f42e8a)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:48.342001 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75afe151-8c90-46af-8f74-8615418e52dd admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:48.351813 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1297/2611] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:12:48 2022] PUT /identity/v3/projects/0d22f3c79d3b4e7e8ce7d102842ea811/users/e1e659e4416044258f1d242ef86b98d0/roles/b43d5a8247e244869c2c70f986f42e8a => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:12:48.361580 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e107520-0a4a-441d-ba58-8220f7d89352 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:48.361895 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e107520-0a4a-441d-ba58-8220f7d89352 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:48.362186 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e107520-0a4a-441d-ba58-8220f7d89352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:48.389048 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-3e107520-0a4a-441d-ba58-8220f7d89352 None None] MFA Rules not processed for user `e1e659e4416044258f1d242ef86b98d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:12:48.390247 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3e107520-0a4a-441d-ba58-8220f7d89352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:12:48.456813 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1315/2612] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:12:48 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:12:48.467717 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75afe151-8c90-46af-8f74-8615418e52dd admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:48.469477 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-75afe151-8c90-46af-8f74-8615418e52dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:12:48.521041 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:12:48.524881 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75afe151-8c90-46af-8f74-8615418e52dd admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:48.528574 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06057dfa-28b4-4e18-b85b-2a30bdb1ed07 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6cc95a563b941ac92ab283aa0d5a5d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:48.529212 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06057dfa-28b4-4e18-b85b-2a30bdb1ed07 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:48.529501 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06057dfa-28b4-4e18-b85b-2a30bdb1ed07 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:48.530039 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06057dfa-28b4-4e18-b85b-2a30bdb1ed07 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:48.538925 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06057dfa-28b4-4e18-b85b-2a30bdb1ed07 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:48.540391 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06057dfa-28b4-4e18-b85b-2a30bdb1ed07 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:48.549688 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1298/2613] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:12:48 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:12:48.696446 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e107520-0a4a-441d-ba58-8220f7d89352 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:48.745629 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:12:48.757738 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e107520-0a4a-441d-ba58-8220f7d89352 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:48.762185 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96987c99-1cf5-4582-b80c-96b4fcbfe716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:48.763099 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96987c99-1cf5-4582-b80c-96b4fcbfe716 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:48.763382 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96987c99-1cf5-4582-b80c-96b4fcbfe716 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:48.763657 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96987c99-1cf5-4582-b80c-96b4fcbfe716 admin admin] PATH_INFO: `/v3/users/6fe75f5f7c3d4aa2b747b2d3b1eb5348` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:48.764857 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96987c99-1cf5-4582-b80c-96b4fcbfe716 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6fe75f5f7c3d4aa2b747b2d3b1eb5348)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:48.766738 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96987c99-1cf5-4582-b80c-96b4fcbfe716 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:48.927108 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-96987c99-1cf5-4582-b80c-96b4fcbfe716 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6fe75f5f7c3d4aa2b747b2d3b1eb5348'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:48.933939 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-96987c99-1cf5-4582-b80c-96b4fcbfe716 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6fe75f5f7c3d4aa2b747b2d3b1eb5348'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:48.999246 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-96987c99-1cf5-4582-b80c-96b4fcbfe716 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6fe75f5f7c3d4aa2b747b2d3b1eb5348'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:48.999905 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-96987c99-1cf5-4582-b80c-96b4fcbfe716 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6fe75f5f7c3d4aa2b747b2d3b1eb5348'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:49.012556 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-96987c99-1cf5-4582-b80c-96b4fcbfe716 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6fe75f5f7c3d4aa2b747b2d3b1eb5348'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:49.019198 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1316/2614] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:12:48 2022] DELETE /identity/v3/users/6fe75f5f7c3d4aa2b747b2d3b1eb5348 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:12:49.023700 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e194a60c-f567-4f78-bdd2-1c162c388947 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:49.024101 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e194a60c-f567-4f78-bdd2-1c162c388947 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:49.024393 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e194a60c-f567-4f78-bdd2-1c162c388947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:49.041777 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-e194a60c-f567-4f78-bdd2-1c162c388947 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:12:49.043020 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e194a60c-f567-4f78-bdd2-1c162c388947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:12:49.106496 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1299/2615] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:12:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:12:49.116110 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96987c99-1cf5-4582-b80c-96b4fcbfe716 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:49.117696 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-96987c99-1cf5-4582-b80c-96b4fcbfe716 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:12:49.174153 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96987c99-1cf5-4582-b80c-96b4fcbfe716 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:49.177860 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7d6d10-902c-4e3d-a7ee-3d07c51e1da3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:49.178526 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7d6d10-902c-4e3d-a7ee-3d07c51e1da3 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:49.178770 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7d6d10-902c-4e3d-a7ee-3d07c51e1da3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:49.179043 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7d6d10-902c-4e3d-a7ee-3d07c51e1da3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:49.188857 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7d6d10-902c-4e3d-a7ee-3d07c51e1da3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:49.190265 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7d6d10-902c-4e3d-a7ee-3d07c51e1da3 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:49.200026 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1317/2616] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:12:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:12:49.592513 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e194a60c-f567-4f78-bdd2-1c162c388947 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:49.595273 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e194a60c-f567-4f78-bdd2-1c162c388947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:12:49.609178 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e194a60c-f567-4f78-bdd2-1c162c388947 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:49.614082 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edc37f6-73cd-4295-b09e-ab98f8428673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:49.615037 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edc37f6-73cd-4295-b09e-ab98f8428673 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:49.615357 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edc37f6-73cd-4295-b09e-ab98f8428673 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:49.615623 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edc37f6-73cd-4295-b09e-ab98f8428673 admin admin] PATH_INFO: `/v3/projects/f5b7f29801c546abb7a89068edf12cd7` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:49.616884 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edc37f6-73cd-4295-b09e-ab98f8428673 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5b7f29801c546abb7a89068edf12cd7)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:49.618794 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edc37f6-73cd-4295-b09e-ab98f8428673 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:49.728273 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4edc37f6-73cd-4295-b09e-ab98f8428673 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5b7f29801c546abb7a89068edf12cd7', 'request_id': 'req-4edc37f6-73cd-4295-b09e-ab98f8428673'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:49.736507 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4edc37f6-73cd-4295-b09e-ab98f8428673 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5b7f29801c546abb7a89068edf12cd7', 'request_id': 'req-4edc37f6-73cd-4295-b09e-ab98f8428673'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:49.746677 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4edc37f6-73cd-4295-b09e-ab98f8428673 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5b7f29801c546abb7a89068edf12cd7', 'request_id': 'req-4edc37f6-73cd-4295-b09e-ab98f8428673'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:49.820569 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4edc37f6-73cd-4295-b09e-ab98f8428673 admin admin] The token cache is being invalidate because project f5b7f29801c546abb7a89068edf12cd7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:12:49.820858 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4edc37f6-73cd-4295-b09e-ab98f8428673 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:49.821587 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-4edc37f6-73cd-4295-b09e-ab98f8428673 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:49.823202 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1300/2617] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:12:49 2022] DELETE /identity/v3/projects/f5b7f29801c546abb7a89068edf12cd7 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:12:49.827236 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7d6d10-902c-4e3d-a7ee-3d07c51e1da3 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:49.836686 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7d6d10-902c-4e3d-a7ee-3d07c51e1da3 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:49.905713 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec33ae9-6c62-4fdc-b6be-510a24ca4ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:49.906638 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec33ae9-6c62-4fdc-b6be-510a24ca4ea3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:49.906926 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec33ae9-6c62-4fdc-b6be-510a24ca4ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:49.907208 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec33ae9-6c62-4fdc-b6be-510a24ca4ea3 admin admin] PATH_INFO: `/v3/users/e1e659e4416044258f1d242ef86b98d0` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:49.908426 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec33ae9-6c62-4fdc-b6be-510a24ca4ea3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1e659e4416044258f1d242ef86b98d0)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:49.910312 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec33ae9-6c62-4fdc-b6be-510a24ca4ea3 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:49.925094 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:12:50.144687 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-eec33ae9-6c62-4fdc-b6be-510a24ca4ea3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1e659e4416044258f1d242ef86b98d0'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:50.151680 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-eec33ae9-6c62-4fdc-b6be-510a24ca4ea3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1e659e4416044258f1d242ef86b98d0'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:50.227643 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-eec33ae9-6c62-4fdc-b6be-510a24ca4ea3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1e659e4416044258f1d242ef86b98d0'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:50.228298 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-eec33ae9-6c62-4fdc-b6be-510a24ca4ea3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1e659e4416044258f1d242ef86b98d0'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:50.240921 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-eec33ae9-6c62-4fdc-b6be-510a24ca4ea3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1e659e4416044258f1d242ef86b98d0'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:50.249628 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1318/2618] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:12:49 2022] DELETE /identity/v3/users/e1e659e4416044258f1d242ef86b98d0 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:12:50.652478 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edc37f6-73cd-4295-b09e-ab98f8428673 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:50.653951 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-4edc37f6-73cd-4295-b09e-ab98f8428673 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:12:50.667497 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edc37f6-73cd-4295-b09e-ab98f8428673 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:50.725611 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9304f4-6142-4c3d-90f7-fd44e0cd48ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:50.726480 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9304f4-6142-4c3d-90f7-fd44e0cd48ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:50.726830 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9304f4-6142-4c3d-90f7-fd44e0cd48ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:50.727121 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9304f4-6142-4c3d-90f7-fd44e0cd48ab admin admin] PATH_INFO: `/v3/projects/0d22f3c79d3b4e7e8ce7d102842ea811` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:50.728441 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9304f4-6142-4c3d-90f7-fd44e0cd48ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d22f3c79d3b4e7e8ce7d102842ea811)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:50.730335 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9304f4-6142-4c3d-90f7-fd44e0cd48ab admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:50.746700 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:12:50.844326 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-7d9304f4-6142-4c3d-90f7-fd44e0cd48ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d22f3c79d3b4e7e8ce7d102842ea811', 'request_id': 'req-7d9304f4-6142-4c3d-90f7-fd44e0cd48ab'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:50.849545 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-7d9304f4-6142-4c3d-90f7-fd44e0cd48ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d22f3c79d3b4e7e8ce7d102842ea811', 'request_id': 'req-7d9304f4-6142-4c3d-90f7-fd44e0cd48ab'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:50.855628 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-7d9304f4-6142-4c3d-90f7-fd44e0cd48ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d22f3c79d3b4e7e8ce7d102842ea811', 'request_id': 'req-7d9304f4-6142-4c3d-90f7-fd44e0cd48ab'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:50.918532 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-7d9304f4-6142-4c3d-90f7-fd44e0cd48ab admin admin] The token cache is being invalidate because project 0d22f3c79d3b4e7e8ce7d102842ea811 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:12:50.919427 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-7d9304f4-6142-4c3d-90f7-fd44e0cd48ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:50.920078 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-7d9304f4-6142-4c3d-90f7-fd44e0cd48ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:50.921683 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1301/2619] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:12:50 2022] DELETE /identity/v3/projects/0d22f3c79d3b4e7e8ce7d102842ea811 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:12:50.926799 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8578f1e9-8447-44b5-a83f-944068cc4241 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:50.927270 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8578f1e9-8447-44b5-a83f-944068cc4241 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:50.927448 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8578f1e9-8447-44b5-a83f-944068cc4241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:50.947545 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-8578f1e9-8447-44b5-a83f-944068cc4241 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:12:50.948686 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8578f1e9-8447-44b5-a83f-944068cc4241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:12:51.007572 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1319/2620] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:12:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:12:51.020669 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338da1d1-4cc2-4137-b8ba-b0544993adcf None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:51.020963 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338da1d1-4cc2-4137-b8ba-b0544993adcf None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:51.021247 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338da1d1-4cc2-4137-b8ba-b0544993adcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:51.039386 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-338da1d1-4cc2-4137-b8ba-b0544993adcf None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:12:51.040616 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-338da1d1-4cc2-4137-b8ba-b0544993adcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:12:51.049162 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1302/2621] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:12:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:12:51.053303 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8578f1e9-8447-44b5-a83f-944068cc4241 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:51.063851 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8578f1e9-8447-44b5-a83f-944068cc4241 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:51.068620 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d11eee-eb18-4f4b-8aab-4d4889dc4b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:51.069272 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d11eee-eb18-4f4b-8aab-4d4889dc4b79 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:51.069657 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d11eee-eb18-4f4b-8aab-4d4889dc4b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:51.069841 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d11eee-eb18-4f4b-8aab-4d4889dc4b79 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:51.070529 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d11eee-eb18-4f4b-8aab-4d4889dc4b79 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 13:12:51.070865 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d11eee-eb18-4f4b-8aab-4d4889dc4b79 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:51.072478 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d11eee-eb18-4f4b-8aab-4d4889dc4b79 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:51.084304 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1320/2622] 10.222.0.27 () {58 vars in 1152 bytes} [Fri Jun 10 13:12:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 13:12:51.088455 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338da1d1-4cc2-4137-b8ba-b0544993adcf None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:51.097596 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338da1d1-4cc2-4137-b8ba-b0544993adcf None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:51.102188 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbdadfe-bd05-4bd0-96f9-05412a3e27a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:51.102835 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbdadfe-bd05-4bd0-96f9-05412a3e27a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:51.103120 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbdadfe-bd05-4bd0-96f9-05412a3e27a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:51.103401 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbdadfe-bd05-4bd0-96f9-05412a3e27a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:51.104128 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbdadfe-bd05-4bd0-96f9-05412a3e27a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:51.106025 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbdadfe-bd05-4bd0-96f9-05412a3e27a5 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:51.223992 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1303/2623] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:12:51 2022] POST /identity/v3/projects => generated 355 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:12:51.228008 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d11eee-eb18-4f4b-8aab-4d4889dc4b79 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:51.238394 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d11eee-eb18-4f4b-8aab-4d4889dc4b79 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:51.293065 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d67cc8-ee08-45fa-a21f-15c9f5580539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:51.293726 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d67cc8-ee08-45fa-a21f-15c9f5580539 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:51.294563 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d67cc8-ee08-45fa-a21f-15c9f5580539 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:51.294852 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d67cc8-ee08-45fa-a21f-15c9f5580539 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:51.295535 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d67cc8-ee08-45fa-a21f-15c9f5580539 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:51.297531 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d67cc8-ee08-45fa-a21f-15c9f5580539 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:51.359206 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1321/2624] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:12:51 2022] POST /identity/v3/users => generated 278 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 10 13:12:51.363186 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbdadfe-bd05-4bd0-96f9-05412a3e27a5 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:51.373391 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbdadfe-bd05-4bd0-96f9-05412a3e27a5 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:51.377961 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb88d39-dae4-4e34-a02e-29a6c84f9523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:51.378615 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb88d39-dae4-4e34-a02e-29a6c84f9523 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:51.378897 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb88d39-dae4-4e34-a02e-29a6c84f9523 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:51.379199 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb88d39-dae4-4e34-a02e-29a6c84f9523 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:51.379954 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb88d39-dae4-4e34-a02e-29a6c84f9523 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:51.381531 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb88d39-dae4-4e34-a02e-29a6c84f9523 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:51.392807 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1304/2625] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:12:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:12:51.396707 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d67cc8-ee08-45fa-a21f-15c9f5580539 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:51.407199 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d67cc8-ee08-45fa-a21f-15c9f5580539 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:51.411694 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6171e8b-fec0-4b12-ba60-56d71ad0d2bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:51.412365 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6171e8b-fec0-4b12-ba60-56d71ad0d2bb admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:51.412645 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6171e8b-fec0-4b12-ba60-56d71ad0d2bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:51.412921 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6171e8b-fec0-4b12-ba60-56d71ad0d2bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:51.413633 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6171e8b-fec0-4b12-ba60-56d71ad0d2bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:51.415186 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6171e8b-fec0-4b12-ba60-56d71ad0d2bb admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:51.417172 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:12:51.426453 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1322/2626] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:12:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:12:51.430793 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb88d39-dae4-4e34-a02e-29a6c84f9523 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:51.440717 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb88d39-dae4-4e34-a02e-29a6c84f9523 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:51.445300 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f54155-fe54-435d-9a75-7bea3ea293a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:51.446195 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f54155-fe54-435d-9a75-7bea3ea293a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:51.446472 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f54155-fe54-435d-9a75-7bea3ea293a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:51.446742 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f54155-fe54-435d-9a75-7bea3ea293a5 admin admin] PATH_INFO: `/v3/projects/26a9ca53e501410188f9169089ab1170/users/26860c2f6c3842a798d32a2a2365fbfc/roles/43106c664e1b40de984c0c62701149c8` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:51.461319 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f54155-fe54-435d-9a75-7bea3ea293a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26a9ca53e501410188f9169089ab1170, user_id=26860c2f6c3842a798d32a2a2365fbfc, role_id=43106c664e1b40de984c0c62701149c8)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:51.464050 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f54155-fe54-435d-9a75-7bea3ea293a5 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:51.512662 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1305/2627] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:12:51 2022] PUT /identity/v3/projects/26a9ca53e501410188f9169089ab1170/users/26860c2f6c3842a798d32a2a2365fbfc/roles/43106c664e1b40de984c0c62701149c8 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:12:51.523428 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951422f2-217c-4cc1-8df9-36c66d7ea641 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:51.523825 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951422f2-217c-4cc1-8df9-36c66d7ea641 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:51.524161 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951422f2-217c-4cc1-8df9-36c66d7ea641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:51.549774 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-951422f2-217c-4cc1-8df9-36c66d7ea641 None None] MFA Rules not processed for user `26860c2f6c3842a798d32a2a2365fbfc`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:12:51.550986 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-951422f2-217c-4cc1-8df9-36c66d7ea641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:12:51.618353 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1323/2628] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:12:51 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:12:51.623504 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f54155-fe54-435d-9a75-7bea3ea293a5 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:51.686659 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f54155-fe54-435d-9a75-7bea3ea293a5 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:51.692151 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43de3c9f-f90b-4ded-8d75-d091256b21d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:51.692811 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43de3c9f-f90b-4ded-8d75-d091256b21d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:51.693093 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43de3c9f-f90b-4ded-8d75-d091256b21d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:51.693369 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43de3c9f-f90b-4ded-8d75-d091256b21d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:51.694066 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43de3c9f-f90b-4ded-8d75-d091256b21d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:51.696084 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43de3c9f-f90b-4ded-8d75-d091256b21d4 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:51.789560 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1306/2629] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:12:51 2022] POST /identity/v3/projects => generated 357 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:12:51.793442 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951422f2-217c-4cc1-8df9-36c66d7ea641 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:51.804445 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951422f2-217c-4cc1-8df9-36c66d7ea641 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:51.860486 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78efbfc4-f282-4d52-affa-9446e0fc9589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:51.860486 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78efbfc4-f282-4d52-affa-9446e0fc9589 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:51.860733 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78efbfc4-f282-4d52-affa-9446e0fc9589 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:51.861037 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78efbfc4-f282-4d52-affa-9446e0fc9589 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:51.861585 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78efbfc4-f282-4d52-affa-9446e0fc9589 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:51.863660 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78efbfc4-f282-4d52-affa-9446e0fc9589 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:51.879746 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43de3c9f-f90b-4ded-8d75-d091256b21d4 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:51.881599 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-43de3c9f-f90b-4ded-8d75-d091256b21d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:12:51.894254 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43de3c9f-f90b-4ded-8d75-d091256b21d4 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:51.899015 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b61425d-afd6-4de6-a79b-209bf9a3bef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:51.900092 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b61425d-afd6-4de6-a79b-209bf9a3bef1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:51.900242 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b61425d-afd6-4de6-a79b-209bf9a3bef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:51.900471 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b61425d-afd6-4de6-a79b-209bf9a3bef1 admin admin] PATH_INFO: `/v3/users/4fdd3e19e7884696ac1fbeb6ba793165` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:51.901654 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b61425d-afd6-4de6-a79b-209bf9a3bef1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4fdd3e19e7884696ac1fbeb6ba793165)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:51.903568 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b61425d-afd6-4de6-a79b-209bf9a3bef1 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:51.925680 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:12:51.969114 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1324/2630] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:12:51 2022] POST /identity/v3/users => generated 285 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:12:51.970534 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78efbfc4-f282-4d52-affa-9446e0fc9589 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:51.980797 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78efbfc4-f282-4d52-affa-9446e0fc9589 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:51.985984 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc64fd16-b08d-4957-bda4-8846ec87099e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:51.986745 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc64fd16-b08d-4957-bda4-8846ec87099e admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:51.987031 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc64fd16-b08d-4957-bda4-8846ec87099e admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:51.987309 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc64fd16-b08d-4957-bda4-8846ec87099e admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:51.988061 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc64fd16-b08d-4957-bda4-8846ec87099e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:51.990031 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc64fd16-b08d-4957-bda4-8846ec87099e admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:52.000904 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1325/2631] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:12:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 10 13:12:52.005107 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc64fd16-b08d-4957-bda4-8846ec87099e admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:52.014983 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc64fd16-b08d-4957-bda4-8846ec87099e admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:52.019684 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d53365-867b-4b2d-9b08-e5ab2c0dc98a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:52.020602 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d53365-867b-4b2d-9b08-e5ab2c0dc98a admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:52.020931 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d53365-867b-4b2d-9b08-e5ab2c0dc98a admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:52.021240 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d53365-867b-4b2d-9b08-e5ab2c0dc98a admin admin] PATH_INFO: `/v3/projects/dfcdfc46110a4a89b1e55754cba17290/users/243bcc55dd504d2cb8ff8602b9108926/roles/b43d5a8247e244869c2c70f986f42e8a` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:52.036894 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d53365-867b-4b2d-9b08-e5ab2c0dc98a admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfcdfc46110a4a89b1e55754cba17290, user_id=243bcc55dd504d2cb8ff8602b9108926, role_id=b43d5a8247e244869c2c70f986f42e8a)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:52.039652 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d53365-867b-4b2d-9b08-e5ab2c0dc98a admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:52.069501 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-9b61425d-afd6-4de6-a79b-209bf9a3bef1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4fdd3e19e7884696ac1fbeb6ba793165'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:52.076207 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-9b61425d-afd6-4de6-a79b-209bf9a3bef1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4fdd3e19e7884696ac1fbeb6ba793165'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:52.089959 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1326/2632] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:12:52 2022] PUT /identity/v3/projects/dfcdfc46110a4a89b1e55754cba17290/users/243bcc55dd504d2cb8ff8602b9108926/roles/b43d5a8247e244869c2c70f986f42e8a => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 10 13:12:52.094324 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d53365-867b-4b2d-9b08-e5ab2c0dc98a admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:52.137400 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:12:52.141771 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-9b61425d-afd6-4de6-a79b-209bf9a3bef1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4fdd3e19e7884696ac1fbeb6ba793165'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:52.142409 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-9b61425d-afd6-4de6-a79b-209bf9a3bef1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4fdd3e19e7884696ac1fbeb6ba793165'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:52.156049 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d53365-867b-4b2d-9b08-e5ab2c0dc98a admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:52.156497 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-9b61425d-afd6-4de6-a79b-209bf9a3bef1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4fdd3e19e7884696ac1fbeb6ba793165'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:52.161761 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5174e87-e885-4716-a17a-15906e8d13bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:52.162460 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5174e87-e885-4716-a17a-15906e8d13bd admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:52.162791 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5174e87-e885-4716-a17a-15906e8d13bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:52.163082 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5174e87-e885-4716-a17a-15906e8d13bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:52.163363 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1307/2633] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:12:51 2022] DELETE /identity/v3/users/4fdd3e19e7884696ac1fbeb6ba793165 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 10 13:12:52.166636 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5174e87-e885-4716-a17a-15906e8d13bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:52.167643 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1af2579-d180-434a-be21-e99800a3000c None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:52.167971 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1af2579-d180-434a-be21-e99800a3000c None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:52.168276 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5174e87-e885-4716-a17a-15906e8d13bd admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:52.168378 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1af2579-d180-434a-be21-e99800a3000c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:52.180889 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1327/2634] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:12:52 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 10 13:12:52.186249 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-a1af2579-d180-434a-be21-e99800a3000c None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:12:52.187821 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a1af2579-d180-434a-be21-e99800a3000c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:12:52.190717 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5174e87-e885-4716-a17a-15906e8d13bd admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:52.192458 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b5174e87-e885-4716-a17a-15906e8d13bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:12:52.204983 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5174e87-e885-4716-a17a-15906e8d13bd admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:52.249179 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1308/2635] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:12:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 10 13:12:52.258809 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1af2579-d180-434a-be21-e99800a3000c None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:52.260183 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a1af2579-d180-434a-be21-e99800a3000c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:12:52.265653 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbfa830-2608-48ab-9874-51b7ef94ace4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:52.266545 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbfa830-2608-48ab-9874-51b7ef94ace4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:52.266826 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbfa830-2608-48ab-9874-51b7ef94ace4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:52.267103 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbfa830-2608-48ab-9874-51b7ef94ace4 admin admin] PATH_INFO: `/v3/projects/dfcdfc46110a4a89b1e55754cba17290/users/243bcc55dd504d2cb8ff8602b9108926/roles/b43d5a8247e244869c2c70f986f42e8a` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:52.269070 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbfa830-2608-48ab-9874-51b7ef94ace4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfcdfc46110a4a89b1e55754cba17290, user_id=243bcc55dd504d2cb8ff8602b9108926, role_id=b43d5a8247e244869c2c70f986f42e8a)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:52.271670 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbfa830-2608-48ab-9874-51b7ef94ace4 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:52.282504 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1328/2636] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:12:52 2022] PUT /identity/v3/projects/dfcdfc46110a4a89b1e55754cba17290/users/243bcc55dd504d2cb8ff8602b9108926/roles/b43d5a8247e244869c2c70f986f42e8a => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 10 13:12:52.296549 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae48ad6-d15f-4c86-842d-528b224f8a16 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:52.296778 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae48ad6-d15f-4c86-842d-528b224f8a16 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:52.297066 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae48ad6-d15f-4c86-842d-528b224f8a16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:52.322879 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1af2579-d180-434a-be21-e99800a3000c None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:52.333561 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-4ae48ad6-d15f-4c86-842d-528b224f8a16 None None] MFA Rules not processed for user `243bcc55dd504d2cb8ff8602b9108926`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:12:52.334766 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-4ae48ad6-d15f-4c86-842d-528b224f8a16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:12:52.372787 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791aff3f-39ae-4f5a-b0f4-49fc2c48bfbb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:52.373455 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791aff3f-39ae-4f5a-b0f4-49fc2c48bfbb service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:52.373720 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791aff3f-39ae-4f5a-b0f4-49fc2c48bfbb service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:52.374001 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791aff3f-39ae-4f5a-b0f4-49fc2c48bfbb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:52.382421 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:12:52.408172 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1329/2637] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:12:52 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 10 13:12:52.419222 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae48ad6-d15f-4c86-842d-528b224f8a16 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:52.420637 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-4ae48ad6-d15f-4c86-842d-528b224f8a16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:12:52.434642 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791aff3f-39ae-4f5a-b0f4-49fc2c48bfbb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:52.436109 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791aff3f-39ae-4f5a-b0f4-49fc2c48bfbb service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:52.451924 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1309/2638] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:12:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 10 13:12:52.481233 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae48ad6-d15f-4c86-842d-528b224f8a16 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:52.484912 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fdfa1c-8d8b-4d50-891d-28b1a5264e0d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6cc95a563b941ac92ab283aa0d5a5d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:52.485568 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fdfa1c-8d8b-4d50-891d-28b1a5264e0d service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:52.485851 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fdfa1c-8d8b-4d50-891d-28b1a5264e0d service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:52.486128 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fdfa1c-8d8b-4d50-891d-28b1a5264e0d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:52.495172 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fdfa1c-8d8b-4d50-891d-28b1a5264e0d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:52.496640 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fdfa1c-8d8b-4d50-891d-28b1a5264e0d service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:52.504863 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:12:52.510223 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1330/2639] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:12:52 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 10 13:12:52.733486 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791aff3f-39ae-4f5a-b0f4-49fc2c48bfbb service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:52.743258 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791aff3f-39ae-4f5a-b0f4-49fc2c48bfbb service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:52.746819 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8be2ec8-b885-489d-a2b9-2859fbd4b9f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6cc95a563b941ac92ab283aa0d5a5d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:52.747998 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8be2ec8-b885-489d-a2b9-2859fbd4b9f4 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:52.748286 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8be2ec8-b885-489d-a2b9-2859fbd4b9f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:52.748862 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8be2ec8-b885-489d-a2b9-2859fbd4b9f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:52.750352 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a8be2ec8-b885-489d-a2b9-2859fbd4b9f4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:12:52.810493 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8be2ec8-b885-489d-a2b9-2859fbd4b9f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:52.811949 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8be2ec8-b885-489d-a2b9-2859fbd4b9f4 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:52.830904 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1310/2640] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:12:52 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:12:52.831469 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fdfa1c-8d8b-4d50-891d-28b1a5264e0d service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:52.831469 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-83fdfa1c-8d8b-4d50-891d-28b1a5264e0d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:12:52.850302 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fdfa1c-8d8b-4d50-891d-28b1a5264e0d service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:52.858901 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa43830f-5837-419b-995e-637e3b019091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:52.859767 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa43830f-5837-419b-995e-637e3b019091 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:52.860076 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa43830f-5837-419b-995e-637e3b019091 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:52.860368 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa43830f-5837-419b-995e-637e3b019091 admin admin] PATH_INFO: `/v3/projects/a9769b685d9942439293d550ee5d1f3b` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:52.861565 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa43830f-5837-419b-995e-637e3b019091 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9769b685d9942439293d550ee5d1f3b)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:52.864487 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa43830f-5837-419b-995e-637e3b019091 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:52.987629 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-fa43830f-5837-419b-995e-637e3b019091 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9769b685d9942439293d550ee5d1f3b', 'request_id': 'req-fa43830f-5837-419b-995e-637e3b019091'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:52.993617 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-fa43830f-5837-419b-995e-637e3b019091 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9769b685d9942439293d550ee5d1f3b', 'request_id': 'req-fa43830f-5837-419b-995e-637e3b019091'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:53.000340 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-fa43830f-5837-419b-995e-637e3b019091 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9769b685d9942439293d550ee5d1f3b', 'request_id': 'req-fa43830f-5837-419b-995e-637e3b019091'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:53.080216 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-fa43830f-5837-419b-995e-637e3b019091 admin admin] The token cache is being invalidate because project a9769b685d9942439293d550ee5d1f3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:12:53.080524 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-fa43830f-5837-419b-995e-637e3b019091 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:53.081150 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-fa43830f-5837-419b-995e-637e3b019091 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:53.082693 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1331/2641] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:12:52 2022] DELETE /identity/v3/projects/a9769b685d9942439293d550ee5d1f3b => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:12:53.086600 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8be2ec8-b885-489d-a2b9-2859fbd4b9f4 service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:53.097149 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8be2ec8-b885-489d-a2b9-2859fbd4b9f4 service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:53.152729 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f639c08b-425b-4a25-a212-69f857c6a032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:53.153648 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f639c08b-425b-4a25-a212-69f857c6a032 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:53.153932 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f639c08b-425b-4a25-a212-69f857c6a032 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:53.154210 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f639c08b-425b-4a25-a212-69f857c6a032 admin admin] PATH_INFO: `/v3/users/69a7f8b327fa4fcdabbdf7f7c19b219a` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:53.155640 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f639c08b-425b-4a25-a212-69f857c6a032 admin admin] RBAC: Authorizing `identity:delete_user(user_id=69a7f8b327fa4fcdabbdf7f7c19b219a)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:53.157586 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f639c08b-425b-4a25-a212-69f857c6a032 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:53.345184 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:12:53.346527 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f639c08b-425b-4a25-a212-69f857c6a032 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69a7f8b327fa4fcdabbdf7f7c19b219a'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:53.354380 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f639c08b-425b-4a25-a212-69f857c6a032 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69a7f8b327fa4fcdabbdf7f7c19b219a'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:53.422562 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f639c08b-425b-4a25-a212-69f857c6a032 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69a7f8b327fa4fcdabbdf7f7c19b219a'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:53.423185 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f639c08b-425b-4a25-a212-69f857c6a032 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69a7f8b327fa4fcdabbdf7f7c19b219a'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:53.436142 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f639c08b-425b-4a25-a212-69f857c6a032 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69a7f8b327fa4fcdabbdf7f7c19b219a'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:53.442646 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1311/2642] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:12:53 2022] DELETE /identity/v3/users/69a7f8b327fa4fcdabbdf7f7c19b219a => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:12:53.943611 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa43830f-5837-419b-995e-637e3b019091 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:53.945121 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-fa43830f-5837-419b-995e-637e3b019091 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:12:53.960107 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa43830f-5837-419b-995e-637e3b019091 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:54.017996 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1e396f-936e-4491-9442-f0bca0ad5647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:54.018843 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1e396f-936e-4491-9442-f0bca0ad5647 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:54.019122 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1e396f-936e-4491-9442-f0bca0ad5647 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:54.019394 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1e396f-936e-4491-9442-f0bca0ad5647 admin admin] PATH_INFO: `/v3/projects/e354352fea384370bb5a89fed919ab9d` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:54.020571 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1e396f-936e-4491-9442-f0bca0ad5647 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e354352fea384370bb5a89fed919ab9d)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:54.022439 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1e396f-936e-4491-9442-f0bca0ad5647 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:54.124045 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-bd1e396f-936e-4491-9442-f0bca0ad5647 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e354352fea384370bb5a89fed919ab9d', 'request_id': 'req-bd1e396f-936e-4491-9442-f0bca0ad5647'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:54.129393 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-bd1e396f-936e-4491-9442-f0bca0ad5647 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e354352fea384370bb5a89fed919ab9d', 'request_id': 'req-bd1e396f-936e-4491-9442-f0bca0ad5647'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:54.135370 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-bd1e396f-936e-4491-9442-f0bca0ad5647 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e354352fea384370bb5a89fed919ab9d', 'request_id': 'req-bd1e396f-936e-4491-9442-f0bca0ad5647'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:54.221785 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-bd1e396f-936e-4491-9442-f0bca0ad5647 admin admin] The token cache is being invalidate because project e354352fea384370bb5a89fed919ab9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:12:54.222007 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-bd1e396f-936e-4491-9442-f0bca0ad5647 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:54.222624 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-bd1e396f-936e-4491-9442-f0bca0ad5647 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:54.224203 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1332/2643] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:12:53 2022] DELETE /identity/v3/projects/e354352fea384370bb5a89fed919ab9d => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:12:54.229049 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da25a1ce-7738-47e4-8ca3-cfdabac662c5 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:54.229558 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da25a1ce-7738-47e4-8ca3-cfdabac662c5 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:54.229672 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da25a1ce-7738-47e4-8ca3-cfdabac662c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:54.249185 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-da25a1ce-7738-47e4-8ca3-cfdabac662c5 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:12:54.250382 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-da25a1ce-7738-47e4-8ca3-cfdabac662c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:12:54.307548 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1312/2644] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:12:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:12:54.317091 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3b4669-a0d8-47a0-94d7-40e3ae29b500 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:54.317386 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3b4669-a0d8-47a0-94d7-40e3ae29b500 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:54.317669 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3b4669-a0d8-47a0-94d7-40e3ae29b500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:54.335845 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-bb3b4669-a0d8-47a0-94d7-40e3ae29b500 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:12:54.337142 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-bb3b4669-a0d8-47a0-94d7-40e3ae29b500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:12:54.344870 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1333/2645] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:12:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:12:54.348874 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da25a1ce-7738-47e4-8ca3-cfdabac662c5 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:54.358253 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da25a1ce-7738-47e4-8ca3-cfdabac662c5 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:54.362734 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc605f2-62bb-400f-b174-3f8dd76f8431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:54.363400 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc605f2-62bb-400f-b174-3f8dd76f8431 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:54.363681 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc605f2-62bb-400f-b174-3f8dd76f8431 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:54.363991 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc605f2-62bb-400f-b174-3f8dd76f8431 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:54.364711 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc605f2-62bb-400f-b174-3f8dd76f8431 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 13:12:54.365009 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc605f2-62bb-400f-b174-3f8dd76f8431 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:54.367467 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc605f2-62bb-400f-b174-3f8dd76f8431 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:54.379578 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1313/2646] 10.222.0.27 () {58 vars in 1152 bytes} [Fri Jun 10 13:12:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 13:12:54.383672 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3b4669-a0d8-47a0-94d7-40e3ae29b500 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:54.393337 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3b4669-a0d8-47a0-94d7-40e3ae29b500 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:54.398132 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2154b87b-ca85-42c6-b985-4a76f268b308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:54.398846 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2154b87b-ca85-42c6-b985-4a76f268b308 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:54.399196 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2154b87b-ca85-42c6-b985-4a76f268b308 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:54.399537 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2154b87b-ca85-42c6-b985-4a76f268b308 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:54.400398 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2154b87b-ca85-42c6-b985-4a76f268b308 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:54.402321 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2154b87b-ca85-42c6-b985-4a76f268b308 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:54.406276 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:12:54.531948 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1334/2647] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:12:54 2022] POST /identity/v3/projects => generated 371 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:12:54.536000 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc605f2-62bb-400f-b174-3f8dd76f8431 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:54.545989 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc605f2-62bb-400f-b174-3f8dd76f8431 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:54.601369 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09d3338-685c-4ce8-8c87-a646bca0bede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:54.602042 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09d3338-685c-4ce8-8c87-a646bca0bede admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:54.602329 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09d3338-685c-4ce8-8c87-a646bca0bede admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:54.602609 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09d3338-685c-4ce8-8c87-a646bca0bede admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:54.603332 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09d3338-685c-4ce8-8c87-a646bca0bede admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:54.605328 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09d3338-685c-4ce8-8c87-a646bca0bede admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:54.678762 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1314/2648] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:12:54 2022] POST /identity/v3/users => generated 286 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:12:54.682854 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2154b87b-ca85-42c6-b985-4a76f268b308 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:54.693547 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2154b87b-ca85-42c6-b985-4a76f268b308 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:54.698155 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47a4451-d906-4bd5-8d34-733c42c91943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:54.698865 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47a4451-d906-4bd5-8d34-733c42c91943 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:54.700507 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47a4451-d906-4bd5-8d34-733c42c91943 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:54.700507 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47a4451-d906-4bd5-8d34-733c42c91943 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:54.700507 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47a4451-d906-4bd5-8d34-733c42c91943 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:54.701802 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47a4451-d906-4bd5-8d34-733c42c91943 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:54.712363 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1335/2649] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:12:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:12:54.716470 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09d3338-685c-4ce8-8c87-a646bca0bede admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:54.735743 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09d3338-685c-4ce8-8c87-a646bca0bede admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:54.743663 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb263a0-24b4-4136-8391-8c4f0d3059fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:54.744391 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb263a0-24b4-4136-8391-8c4f0d3059fc admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:54.744677 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb263a0-24b4-4136-8391-8c4f0d3059fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:54.744968 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb263a0-24b4-4136-8391-8c4f0d3059fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:54.745685 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb263a0-24b4-4136-8391-8c4f0d3059fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:54.748009 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb263a0-24b4-4136-8391-8c4f0d3059fc admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:54.758516 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1315/2650] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:12:54 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:12:54.762715 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47a4451-d906-4bd5-8d34-733c42c91943 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:54.772581 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47a4451-d906-4bd5-8d34-733c42c91943 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:54.777176 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7a79b7-e6fb-4b5d-a10c-565d8b99b9e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:54.777938 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7a79b7-e6fb-4b5d-a10c-565d8b99b9e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:54.778252 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7a79b7-e6fb-4b5d-a10c-565d8b99b9e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:54.778528 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7a79b7-e6fb-4b5d-a10c-565d8b99b9e9 admin admin] PATH_INFO: `/v3/projects/8d75ec620fc346d79f40e7c9fb7011ac/users/555b1da5ffc8418ba32399ea346417dc/roles/43106c664e1b40de984c0c62701149c8` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:54.793633 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7a79b7-e6fb-4b5d-a10c-565d8b99b9e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d75ec620fc346d79f40e7c9fb7011ac, user_id=555b1da5ffc8418ba32399ea346417dc, role_id=43106c664e1b40de984c0c62701149c8)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:54.796962 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7a79b7-e6fb-4b5d-a10c-565d8b99b9e9 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:54.855444 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1336/2651] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:12:54 2022] PUT /identity/v3/projects/8d75ec620fc346d79f40e7c9fb7011ac/users/555b1da5ffc8418ba32399ea346417dc/roles/43106c664e1b40de984c0c62701149c8 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 10 13:12:54.866375 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd099643-446b-43af-b17a-cee94f4bad72 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:54.866674 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd099643-446b-43af-b17a-cee94f4bad72 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:54.866963 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd099643-446b-43af-b17a-cee94f4bad72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:54.894318 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-dd099643-446b-43af-b17a-cee94f4bad72 None None] MFA Rules not processed for user `555b1da5ffc8418ba32399ea346417dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:12:54.895515 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-dd099643-446b-43af-b17a-cee94f4bad72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:12:54.912680 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:12:54.959904 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1316/2652] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:12:54 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:12:54.972276 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7a79b7-e6fb-4b5d-a10c-565d8b99b9e9 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:54.974154 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6b7a79b7-e6fb-4b5d-a10c-565d8b99b9e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:12:55.030967 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7a79b7-e6fb-4b5d-a10c-565d8b99b9e9 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:55.034748 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e921871-3df0-4607-94d7-324a9782bc89 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6cc95a563b941ac92ab283aa0d5a5d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:55.035425 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e921871-3df0-4607-94d7-324a9782bc89 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:55.035824 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e921871-3df0-4607-94d7-324a9782bc89 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:55.036158 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e921871-3df0-4607-94d7-324a9782bc89 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:55.044558 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e921871-3df0-4607-94d7-324a9782bc89 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:55.045952 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e921871-3df0-4607-94d7-324a9782bc89 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:55.054673 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1337/2653] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:12:54 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:12:55.074419 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd099643-446b-43af-b17a-cee94f4bad72 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:55.135355 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd099643-446b-43af-b17a-cee94f4bad72 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:55.140628 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e202630-faa3-434b-a916-247701bb1954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:55.141504 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e202630-faa3-434b-a916-247701bb1954 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:55.141786 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e202630-faa3-434b-a916-247701bb1954 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:55.142065 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e202630-faa3-434b-a916-247701bb1954 admin admin] PATH_INFO: `/v3/users/555b1da5ffc8418ba32399ea346417dc` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:55.143271 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e202630-faa3-434b-a916-247701bb1954 admin admin] RBAC: Authorizing `identity:delete_user(user_id=555b1da5ffc8418ba32399ea346417dc)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:55.145221 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e202630-faa3-434b-a916-247701bb1954 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:55.366465 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-5e202630-faa3-434b-a916-247701bb1954 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '555b1da5ffc8418ba32399ea346417dc'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:55.374071 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-5e202630-faa3-434b-a916-247701bb1954 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '555b1da5ffc8418ba32399ea346417dc'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:55.438465 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-5e202630-faa3-434b-a916-247701bb1954 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '555b1da5ffc8418ba32399ea346417dc'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:55.439079 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-5e202630-faa3-434b-a916-247701bb1954 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '555b1da5ffc8418ba32399ea346417dc'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:55.451018 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-5e202630-faa3-434b-a916-247701bb1954 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '555b1da5ffc8418ba32399ea346417dc'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:55.456890 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1317/2654] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:12:55 2022] DELETE /identity/v3/users/555b1da5ffc8418ba32399ea346417dc => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:12:55.461391 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb02bfc-082a-4130-931b-14db8369b3d7 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:55.461682 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb02bfc-082a-4130-931b-14db8369b3d7 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:55.461964 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb02bfc-082a-4130-931b-14db8369b3d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:55.478727 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-afb02bfc-082a-4130-931b-14db8369b3d7 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:12:55.479949 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-afb02bfc-082a-4130-931b-14db8369b3d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:12:55.537959 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1338/2655] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:12:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:12:55.547966 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e202630-faa3-434b-a916-247701bb1954 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:55.549414 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5e202630-faa3-434b-a916-247701bb1954 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:12:55.590180 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:12:55.604539 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e202630-faa3-434b-a916-247701bb1954 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:55.608663 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27663a7b-6731-4fe6-9811-83eae9b97989 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:55.609311 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27663a7b-6731-4fe6-9811-83eae9b97989 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:55.609797 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27663a7b-6731-4fe6-9811-83eae9b97989 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:55.610121 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27663a7b-6731-4fe6-9811-83eae9b97989 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:55.621116 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27663a7b-6731-4fe6-9811-83eae9b97989 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:55.621116 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27663a7b-6731-4fe6-9811-83eae9b97989 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:55.630723 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1318/2656] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:12:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 10 13:12:56.040127 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb02bfc-082a-4130-931b-14db8369b3d7 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:56.041615 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-afb02bfc-082a-4130-931b-14db8369b3d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:12:56.054524 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb02bfc-082a-4130-931b-14db8369b3d7 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:56.059848 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfffda8d-e4e2-4528-9a34-b468286acc63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:56.060815 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfffda8d-e4e2-4528-9a34-b468286acc63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:56.061169 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfffda8d-e4e2-4528-9a34-b468286acc63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:56.061455 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfffda8d-e4e2-4528-9a34-b468286acc63 admin admin] PATH_INFO: `/v3/projects/8d75ec620fc346d79f40e7c9fb7011ac` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:56.062726 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfffda8d-e4e2-4528-9a34-b468286acc63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d75ec620fc346d79f40e7c9fb7011ac)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:56.069841 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfffda8d-e4e2-4528-9a34-b468286acc63 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:56.171235 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:12:56.172496 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-bfffda8d-e4e2-4528-9a34-b468286acc63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d75ec620fc346d79f40e7c9fb7011ac', 'request_id': 'req-bfffda8d-e4e2-4528-9a34-b468286acc63'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:56.178517 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-bfffda8d-e4e2-4528-9a34-b468286acc63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d75ec620fc346d79f40e7c9fb7011ac', 'request_id': 'req-bfffda8d-e4e2-4528-9a34-b468286acc63'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:56.185363 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-bfffda8d-e4e2-4528-9a34-b468286acc63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d75ec620fc346d79f40e7c9fb7011ac', 'request_id': 'req-bfffda8d-e4e2-4528-9a34-b468286acc63'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:56.238496 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-bfffda8d-e4e2-4528-9a34-b468286acc63 admin admin] The token cache is being invalidate because project 8d75ec620fc346d79f40e7c9fb7011ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:12:56.238790 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-bfffda8d-e4e2-4528-9a34-b468286acc63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:56.239425 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-bfffda8d-e4e2-4528-9a34-b468286acc63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:12:56.246881 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1339/2657] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:12:56 2022] DELETE /identity/v3/projects/8d75ec620fc346d79f40e7c9fb7011ac => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 10 13:12:56.247001 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931f50b1-e008-4e6a-9df6-736747a500e8 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:56.247001 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931f50b1-e008-4e6a-9df6-736747a500e8 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:56.247001 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931f50b1-e008-4e6a-9df6-736747a500e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:56.265249 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-931f50b1-e008-4e6a-9df6-736747a500e8 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:12:56.266512 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-931f50b1-e008-4e6a-9df6-736747a500e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:12:56.336648 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1319/2658] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:12:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:12:56.346613 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b1e75b-ba07-463c-a948-4014cf915fce None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:56.346907 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b1e75b-ba07-463c-a948-4014cf915fce None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:56.347188 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b1e75b-ba07-463c-a948-4014cf915fce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:56.365180 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-15b1e75b-ba07-463c-a948-4014cf915fce None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:12:56.369514 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-15b1e75b-ba07-463c-a948-4014cf915fce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:12:56.381344 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1340/2659] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:12:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:12:56.385585 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931f50b1-e008-4e6a-9df6-736747a500e8 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:56.404136 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931f50b1-e008-4e6a-9df6-736747a500e8 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:56.404136 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5354e4a-53cb-43a6-b6c2-90038880484a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:56.404136 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5354e4a-53cb-43a6-b6c2-90038880484a admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:56.404136 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5354e4a-53cb-43a6-b6c2-90038880484a admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:56.404541 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5354e4a-53cb-43a6-b6c2-90038880484a admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:56.404541 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5354e4a-53cb-43a6-b6c2-90038880484a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 13:12:56.404541 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5354e4a-53cb-43a6-b6c2-90038880484a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:56.405588 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5354e4a-53cb-43a6-b6c2-90038880484a admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:56.419178 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1320/2660] 10.222.0.27 () {58 vars in 1152 bytes} [Fri Jun 10 13:12:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 13:12:56.426937 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b1e75b-ba07-463c-a948-4014cf915fce None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:56.436157 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b1e75b-ba07-463c-a948-4014cf915fce None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:56.441890 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6598d38b-caea-4615-b06d-d28fbf1e0785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:56.444218 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6598d38b-caea-4615-b06d-d28fbf1e0785 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:56.444218 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6598d38b-caea-4615-b06d-d28fbf1e0785 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:56.444218 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6598d38b-caea-4615-b06d-d28fbf1e0785 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:56.444218 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6598d38b-caea-4615-b06d-d28fbf1e0785 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:56.445733 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6598d38b-caea-4615-b06d-d28fbf1e0785 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:56.526412 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1341/2661] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:12:56 2022] POST /identity/v3/projects => generated 381 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:12:56.531061 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5354e4a-53cb-43a6-b6c2-90038880484a admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:56.540921 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5354e4a-53cb-43a6-b6c2-90038880484a admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:56.597290 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9247f56-d577-4d3f-85c1-4a1acc20047c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:56.597983 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9247f56-d577-4d3f-85c1-4a1acc20047c admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:56.598266 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9247f56-d577-4d3f-85c1-4a1acc20047c admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:56.598540 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9247f56-d577-4d3f-85c1-4a1acc20047c admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:56.599229 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9247f56-d577-4d3f-85c1-4a1acc20047c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:56.601181 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9247f56-d577-4d3f-85c1-4a1acc20047c admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:56.686106 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1321/2662] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:12:56 2022] POST /identity/v3/users => generated 291 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 10 13:12:56.690033 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6598d38b-caea-4615-b06d-d28fbf1e0785 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:56.700914 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6598d38b-caea-4615-b06d-d28fbf1e0785 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:56.705520 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6a75ac-1ee9-417c-a969-cca69fd60a57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:56.706172 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6a75ac-1ee9-417c-a969-cca69fd60a57 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:56.706454 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6a75ac-1ee9-417c-a969-cca69fd60a57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:56.706731 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6a75ac-1ee9-417c-a969-cca69fd60a57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:56.707452 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6a75ac-1ee9-417c-a969-cca69fd60a57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:56.709063 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6a75ac-1ee9-417c-a969-cca69fd60a57 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:56.719222 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1342/2663] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:12:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:12:56.723043 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9247f56-d577-4d3f-85c1-4a1acc20047c admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:56.733229 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9247f56-d577-4d3f-85c1-4a1acc20047c admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:56.738009 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dacc444-c692-4b68-8044-2fd3ac95c4e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:56.738747 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dacc444-c692-4b68-8044-2fd3ac95c4e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:56.739032 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dacc444-c692-4b68-8044-2fd3ac95c4e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:56.739316 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dacc444-c692-4b68-8044-2fd3ac95c4e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:56.740103 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dacc444-c692-4b68-8044-2fd3ac95c4e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:56.741695 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dacc444-c692-4b68-8044-2fd3ac95c4e9 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:56.752384 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1322/2664] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:12:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:12:56.756570 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6a75ac-1ee9-417c-a969-cca69fd60a57 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:56.767198 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6a75ac-1ee9-417c-a969-cca69fd60a57 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:56.771848 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea59353-81dc-4a02-8ffd-d5568b5a84e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:56.772720 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea59353-81dc-4a02-8ffd-d5568b5a84e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:56.773002 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea59353-81dc-4a02-8ffd-d5568b5a84e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:56.773279 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea59353-81dc-4a02-8ffd-d5568b5a84e2 admin admin] PATH_INFO: `/v3/projects/dce47649142f4c60ae08a6f9f1732386/users/7db0e455e2c841b5ae1e9b60f3dd0906/roles/43106c664e1b40de984c0c62701149c8` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:56.788805 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea59353-81dc-4a02-8ffd-d5568b5a84e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dce47649142f4c60ae08a6f9f1732386, user_id=7db0e455e2c841b5ae1e9b60f3dd0906, role_id=43106c664e1b40de984c0c62701149c8)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:56.791394 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea59353-81dc-4a02-8ffd-d5568b5a84e2 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:56.849738 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1343/2665] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:12:56 2022] PUT /identity/v3/projects/dce47649142f4c60ae08a6f9f1732386/users/7db0e455e2c841b5ae1e9b60f3dd0906/roles/43106c664e1b40de984c0c62701149c8 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 10 13:12:56.854782 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41058b6-b9fe-44f8-87ed-ad6f645731c3 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:56.855224 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41058b6-b9fe-44f8-87ed-ad6f645731c3 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:56.855512 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41058b6-b9fe-44f8-87ed-ad6f645731c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:56.875037 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-e41058b6-b9fe-44f8-87ed-ad6f645731c3 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:12:56.876268 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-e41058b6-b9fe-44f8-87ed-ad6f645731c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:12:56.916565 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:12:56.933946 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1323/2666] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:12:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:12:56.944030 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea59353-81dc-4a02-8ffd-d5568b5a84e2 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:56.945796 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-9ea59353-81dc-4a02-8ffd-d5568b5a84e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:12:57.001072 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea59353-81dc-4a02-8ffd-d5568b5a84e2 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:57.004718 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5db13e2-9e9d-44cf-b1e1-ef208efe8dac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:57.005351 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5db13e2-9e9d-44cf-b1e1-ef208efe8dac service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:57.005629 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5db13e2-9e9d-44cf-b1e1-ef208efe8dac service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:57.005902 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5db13e2-9e9d-44cf-b1e1-ef208efe8dac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:57.015207 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5db13e2-9e9d-44cf-b1e1-ef208efe8dac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:57.016694 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5db13e2-9e9d-44cf-b1e1-ef208efe8dac service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:57.026025 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1344/2667] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:12:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:12:58.204845 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c667d1f-ddbe-4f80-9ee2-bb9a09ac1437 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:58.205354 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c667d1f-ddbe-4f80-9ee2-bb9a09ac1437 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:58.205592 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c667d1f-ddbe-4f80-9ee2-bb9a09ac1437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:58.234728 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-0c667d1f-ddbe-4f80-9ee2-bb9a09ac1437 None None] MFA Rules not processed for user `7db0e455e2c841b5ae1e9b60f3dd0906`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:12:58.236093 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0c667d1f-ddbe-4f80-9ee2-bb9a09ac1437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:12:58.303627 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1324/2668] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:12:58 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:12:58.315050 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5db13e2-9e9d-44cf-b1e1-ef208efe8dac service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:12:58.316688 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b5db13e2-9e9d-44cf-b1e1-ef208efe8dac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:12:58.373554 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5db13e2-9e9d-44cf-b1e1-ef208efe8dac service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:12:58.377200 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4939966a-9868-4216-9cef-4faf53ecc6a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6cc95a563b941ac92ab283aa0d5a5d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:12:58.380532 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4939966a-9868-4216-9cef-4faf53ecc6a4 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:12:58.380532 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4939966a-9868-4216-9cef-4faf53ecc6a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:12:58.380532 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4939966a-9868-4216-9cef-4faf53ecc6a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:12:58.387977 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4939966a-9868-4216-9cef-4faf53ecc6a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:12:58.389509 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4939966a-9868-4216-9cef-4faf53ecc6a4 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:12:58.398610 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1345/2669] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:12:58 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:13:00.061780 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464dd4c5-cfcf-42b3-9feb-083a24545461 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:00.062280 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464dd4c5-cfcf-42b3-9feb-083a24545461 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:00.062382 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464dd4c5-cfcf-42b3-9feb-083a24545461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:00.080187 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-464dd4c5-cfcf-42b3-9feb-083a24545461 None None] MFA Rules not processed for user `7db0e455e2c841b5ae1e9b60f3dd0906`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:13:00.081376 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-464dd4c5-cfcf-42b3-9feb-083a24545461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:13:00.088127 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1325/2670] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:13:00 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:13:08.394327 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=86,keystone:DELETE=3,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:13:10.074527 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=41 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:13:10.259140 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4939966a-9868-4216-9cef-4faf53ecc6a4 service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:10.260952 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-4939966a-9868-4216-9cef-4faf53ecc6a4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:13:10.278380 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4939966a-9868-4216-9cef-4faf53ecc6a4 service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:10.283689 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4558576e-4198-4c88-b602-6c8986cdced0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:10.284572 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4558576e-4198-4c88-b602-6c8986cdced0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:10.285326 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4558576e-4198-4c88-b602-6c8986cdced0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:10.285326 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4558576e-4198-4c88-b602-6c8986cdced0 admin admin] PATH_INFO: `/v3/users/26860c2f6c3842a798d32a2a2365fbfc` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:10.286276 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4558576e-4198-4c88-b602-6c8986cdced0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=26860c2f6c3842a798d32a2a2365fbfc)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:10.288221 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4558576e-4198-4c88-b602-6c8986cdced0 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:10.461037 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4558576e-4198-4c88-b602-6c8986cdced0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26860c2f6c3842a798d32a2a2365fbfc'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:10.468983 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4558576e-4198-4c88-b602-6c8986cdced0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26860c2f6c3842a798d32a2a2365fbfc'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:10.531284 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4558576e-4198-4c88-b602-6c8986cdced0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26860c2f6c3842a798d32a2a2365fbfc'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:10.531940 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4558576e-4198-4c88-b602-6c8986cdced0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26860c2f6c3842a798d32a2a2365fbfc'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:10.546939 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4558576e-4198-4c88-b602-6c8986cdced0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26860c2f6c3842a798d32a2a2365fbfc'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:10.553088 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1346/2671] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:13:10 2022] DELETE /identity/v3/users/26860c2f6c3842a798d32a2a2365fbfc => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:13:10.557923 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22f6577-2bbb-45ec-96f5-dc678ef862ad None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:10.558213 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22f6577-2bbb-45ec-96f5-dc678ef862ad None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:10.558492 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22f6577-2bbb-45ec-96f5-dc678ef862ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:10.575954 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-c22f6577-2bbb-45ec-96f5-dc678ef862ad None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:13:10.577668 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c22f6577-2bbb-45ec-96f5-dc678ef862ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:13:10.639785 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1326/2672] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:13:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:13:10.649377 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4558576e-4198-4c88-b602-6c8986cdced0 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:10.650829 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-4558576e-4198-4c88-b602-6c8986cdced0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:13:10.711130 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4558576e-4198-4c88-b602-6c8986cdced0 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:10.714804 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dafd783-efc5-40a3-844e-ca38046f54aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:10.715450 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dafd783-efc5-40a3-844e-ca38046f54aa service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:10.715736 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dafd783-efc5-40a3-844e-ca38046f54aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:10.716145 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dafd783-efc5-40a3-844e-ca38046f54aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:10.726114 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dafd783-efc5-40a3-844e-ca38046f54aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:10.727570 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dafd783-efc5-40a3-844e-ca38046f54aa service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:10.737848 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1347/2673] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:13:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:13:11.364579 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22f6577-2bbb-45ec-96f5-dc678ef862ad None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:11.367257 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c22f6577-2bbb-45ec-96f5-dc678ef862ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:13:11.389278 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22f6577-2bbb-45ec-96f5-dc678ef862ad None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:11.395738 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7878acbd-0dcd-47c4-b00d-5a96a958e39b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:11.397084 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7878acbd-0dcd-47c4-b00d-5a96a958e39b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:11.397624 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7878acbd-0dcd-47c4-b00d-5a96a958e39b admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:11.398006 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7878acbd-0dcd-47c4-b00d-5a96a958e39b admin admin] PATH_INFO: `/v3/projects/26a9ca53e501410188f9169089ab1170` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:11.399817 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7878acbd-0dcd-47c4-b00d-5a96a958e39b admin admin] RBAC: Authorizing `identity:delete_project(project_id=26a9ca53e501410188f9169089ab1170)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:11.401828 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7878acbd-0dcd-47c4-b00d-5a96a958e39b admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:11.502575 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-7878acbd-0dcd-47c4-b00d-5a96a958e39b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26a9ca53e501410188f9169089ab1170', 'request_id': 'req-7878acbd-0dcd-47c4-b00d-5a96a958e39b'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:11.508560 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-7878acbd-0dcd-47c4-b00d-5a96a958e39b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26a9ca53e501410188f9169089ab1170', 'request_id': 'req-7878acbd-0dcd-47c4-b00d-5a96a958e39b'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:11.515847 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-7878acbd-0dcd-47c4-b00d-5a96a958e39b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26a9ca53e501410188f9169089ab1170', 'request_id': 'req-7878acbd-0dcd-47c4-b00d-5a96a958e39b'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:11.575409 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-7878acbd-0dcd-47c4-b00d-5a96a958e39b admin admin] The token cache is being invalidate because project 26a9ca53e501410188f9169089ab1170 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:13:11.575704 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-7878acbd-0dcd-47c4-b00d-5a96a958e39b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:11.577741 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-7878acbd-0dcd-47c4-b00d-5a96a958e39b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:11.579035 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1327/2674] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:13:11 2022] DELETE /identity/v3/projects/26a9ca53e501410188f9169089ab1170 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:13:11.583005 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dafd783-efc5-40a3-844e-ca38046f54aa service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:11.592760 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dafd783-efc5-40a3-844e-ca38046f54aa service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:11.651367 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09db5a8-2e63-4ff5-b279-bb7e1dc11ec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:11.652262 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09db5a8-2e63-4ff5-b279-bb7e1dc11ec2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:11.652547 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09db5a8-2e63-4ff5-b279-bb7e1dc11ec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:11.652824 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09db5a8-2e63-4ff5-b279-bb7e1dc11ec2 admin admin] PATH_INFO: `/v3/users/243bcc55dd504d2cb8ff8602b9108926` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:11.657457 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09db5a8-2e63-4ff5-b279-bb7e1dc11ec2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=243bcc55dd504d2cb8ff8602b9108926)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:11.657457 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09db5a8-2e63-4ff5-b279-bb7e1dc11ec2 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:11.671057 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:13:11.853087 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d09db5a8-2e63-4ff5-b279-bb7e1dc11ec2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '243bcc55dd504d2cb8ff8602b9108926'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:11.861783 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d09db5a8-2e63-4ff5-b279-bb7e1dc11ec2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '243bcc55dd504d2cb8ff8602b9108926'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:11.924665 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d09db5a8-2e63-4ff5-b279-bb7e1dc11ec2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '243bcc55dd504d2cb8ff8602b9108926'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:11.925365 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d09db5a8-2e63-4ff5-b279-bb7e1dc11ec2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '243bcc55dd504d2cb8ff8602b9108926'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:11.940807 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d09db5a8-2e63-4ff5-b279-bb7e1dc11ec2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '243bcc55dd504d2cb8ff8602b9108926'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:11.946836 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1348/2675] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:13:11 2022] DELETE /identity/v3/users/243bcc55dd504d2cb8ff8602b9108926 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:13:12.367295 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7878acbd-0dcd-47c4-b00d-5a96a958e39b admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:12.368151 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-7878acbd-0dcd-47c4-b00d-5a96a958e39b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:13:12.381816 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7878acbd-0dcd-47c4-b00d-5a96a958e39b admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:12.444445 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fd739b-458a-422f-8799-ab4e9771b141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:12.445314 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fd739b-458a-422f-8799-ab4e9771b141 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:12.445601 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fd739b-458a-422f-8799-ab4e9771b141 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:12.445881 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fd739b-458a-422f-8799-ab4e9771b141 admin admin] PATH_INFO: `/v3/projects/dfcdfc46110a4a89b1e55754cba17290` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:12.447117 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fd739b-458a-422f-8799-ab4e9771b141 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dfcdfc46110a4a89b1e55754cba17290)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:12.449098 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fd739b-458a-422f-8799-ab4e9771b141 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:12.549591 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:13:12.565635 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-40fd739b-458a-422f-8799-ab4e9771b141 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dfcdfc46110a4a89b1e55754cba17290', 'request_id': 'req-40fd739b-458a-422f-8799-ab4e9771b141'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:12.571117 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-40fd739b-458a-422f-8799-ab4e9771b141 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dfcdfc46110a4a89b1e55754cba17290', 'request_id': 'req-40fd739b-458a-422f-8799-ab4e9771b141'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:12.577930 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-40fd739b-458a-422f-8799-ab4e9771b141 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dfcdfc46110a4a89b1e55754cba17290', 'request_id': 'req-40fd739b-458a-422f-8799-ab4e9771b141'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:12.678275 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-40fd739b-458a-422f-8799-ab4e9771b141 admin admin] The token cache is being invalidate because project dfcdfc46110a4a89b1e55754cba17290 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:13:12.678499 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-40fd739b-458a-422f-8799-ab4e9771b141 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:12.679112 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-40fd739b-458a-422f-8799-ab4e9771b141 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:12.680705 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1328/2676] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:13:12 2022] DELETE /identity/v3/projects/dfcdfc46110a4a89b1e55754cba17290 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:13:12.685728 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e06c7f5-f906-41a2-b83a-926bd16b3181 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:12.686183 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e06c7f5-f906-41a2-b83a-926bd16b3181 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:12.686315 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e06c7f5-f906-41a2-b83a-926bd16b3181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:12.704143 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-5e06c7f5-f906-41a2-b83a-926bd16b3181 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:13:12.705437 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-5e06c7f5-f906-41a2-b83a-926bd16b3181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:13:12.763497 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1349/2677] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:13:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:13:12.771736 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427a3eb5-b450-4916-9ecc-d74a06a3f4d7 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:12.772090 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427a3eb5-b450-4916-9ecc-d74a06a3f4d7 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:12.772334 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427a3eb5-b450-4916-9ecc-d74a06a3f4d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:12.791497 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-427a3eb5-b450-4916-9ecc-d74a06a3f4d7 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:13:12.796401 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-427a3eb5-b450-4916-9ecc-d74a06a3f4d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:13:12.801373 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1329/2678] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:13:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:13:12.805559 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e06c7f5-f906-41a2-b83a-926bd16b3181 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:12.815634 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e06c7f5-f906-41a2-b83a-926bd16b3181 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:12.820303 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07460fc-e8e6-453f-a348-b632ff0df3f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:12.821057 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07460fc-e8e6-453f-a348-b632ff0df3f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:12.821343 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07460fc-e8e6-453f-a348-b632ff0df3f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:12.821626 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07460fc-e8e6-453f-a348-b632ff0df3f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:12.822355 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07460fc-e8e6-453f-a348-b632ff0df3f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 13:13:12.822665 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07460fc-e8e6-453f-a348-b632ff0df3f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:12.824334 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07460fc-e8e6-453f-a348-b632ff0df3f5 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:12.836107 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1350/2679] 10.222.0.27 () {58 vars in 1152 bytes} [Fri Jun 10 13:13:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 13:13:12.840520 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427a3eb5-b450-4916-9ecc-d74a06a3f4d7 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:12.849621 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427a3eb5-b450-4916-9ecc-d74a06a3f4d7 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:12.854430 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d65dd9-0d6d-4d92-83f7-6219cdf66daa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:12.855094 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d65dd9-0d6d-4d92-83f7-6219cdf66daa admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:12.855382 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d65dd9-0d6d-4d92-83f7-6219cdf66daa admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:12.855780 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d65dd9-0d6d-4d92-83f7-6219cdf66daa admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:12.856518 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d65dd9-0d6d-4d92-83f7-6219cdf66daa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:12.858974 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d65dd9-0d6d-4d92-83f7-6219cdf66daa admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:12.938427 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1330/2680] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:13:12 2022] POST /identity/v3/projects => generated 399 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 10 13:13:12.941786 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07460fc-e8e6-453f-a348-b632ff0df3f5 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:12.951754 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07460fc-e8e6-453f-a348-b632ff0df3f5 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:13.007675 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0aa655-cede-44f0-a448-f04a273d3ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:13.008375 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0aa655-cede-44f0-a448-f04a273d3ec4 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:13.008664 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0aa655-cede-44f0-a448-f04a273d3ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:13.008944 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0aa655-cede-44f0-a448-f04a273d3ec4 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:13.009626 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0aa655-cede-44f0-a448-f04a273d3ec4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:13.011615 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0aa655-cede-44f0-a448-f04a273d3ec4 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:13.073755 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1351/2681] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:13:12 2022] POST /identity/v3/users => generated 300 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:13:13.077253 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d65dd9-0d6d-4d92-83f7-6219cdf66daa admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:13.087610 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d65dd9-0d6d-4d92-83f7-6219cdf66daa admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:13.092220 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19ac518-f1cd-43f3-893e-49d61ca7383a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:13.093379 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19ac518-f1cd-43f3-893e-49d61ca7383a admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:13.093655 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19ac518-f1cd-43f3-893e-49d61ca7383a admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:13.093958 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19ac518-f1cd-43f3-893e-49d61ca7383a admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:13.094641 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19ac518-f1cd-43f3-893e-49d61ca7383a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:13.096251 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19ac518-f1cd-43f3-893e-49d61ca7383a admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:13.106643 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1331/2682] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:13:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:13:13.111067 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0aa655-cede-44f0-a448-f04a273d3ec4 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:13.120797 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0aa655-cede-44f0-a448-f04a273d3ec4 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:13.125346 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ab932c-0df4-45da-8160-c45d4b5071e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:13.125988 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ab932c-0df4-45da-8160-c45d4b5071e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:13.126286 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ab932c-0df4-45da-8160-c45d4b5071e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:13.126550 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ab932c-0df4-45da-8160-c45d4b5071e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:13.127263 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ab932c-0df4-45da-8160-c45d4b5071e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:13.128888 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ab932c-0df4-45da-8160-c45d4b5071e1 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:13.133982 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:13:13.139706 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1352/2683] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:13:13 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:13:13.143550 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19ac518-f1cd-43f3-893e-49d61ca7383a admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:13.153393 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19ac518-f1cd-43f3-893e-49d61ca7383a admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:13.159055 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104873f3-b43b-427b-8d4a-84822fac19a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:13.159972 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104873f3-b43b-427b-8d4a-84822fac19a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:13.160258 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104873f3-b43b-427b-8d4a-84822fac19a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:13.160537 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104873f3-b43b-427b-8d4a-84822fac19a8 admin admin] PATH_INFO: `/v3/projects/b9eab1f46720410cab695b55afc2a1ba/users/cf2aca344ebe4bdfbcf5a6ea6b53d97a/roles/43106c664e1b40de984c0c62701149c8` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:13.175769 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104873f3-b43b-427b-8d4a-84822fac19a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9eab1f46720410cab695b55afc2a1ba, user_id=cf2aca344ebe4bdfbcf5a6ea6b53d97a, role_id=43106c664e1b40de984c0c62701149c8)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:13.179641 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104873f3-b43b-427b-8d4a-84822fac19a8 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:13.256182 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1332/2684] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:13:13 2022] PUT /identity/v3/projects/b9eab1f46720410cab695b55afc2a1ba/users/cf2aca344ebe4bdfbcf5a6ea6b53d97a/roles/43106c664e1b40de984c0c62701149c8 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:13:13.267229 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1755a864-444b-455c-bf8f-1b5255647381 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:13.267780 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1755a864-444b-455c-bf8f-1b5255647381 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:13.268286 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1755a864-444b-455c-bf8f-1b5255647381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:13.294699 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-1755a864-444b-455c-bf8f-1b5255647381 None None] MFA Rules not processed for user `cf2aca344ebe4bdfbcf5a6ea6b53d97a`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:13:13.295934 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1755a864-444b-455c-bf8f-1b5255647381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:13:13.364926 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1353/2685] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:13:13 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:13:13.369340 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104873f3-b43b-427b-8d4a-84822fac19a8 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:13.435576 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104873f3-b43b-427b-8d4a-84822fac19a8 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:13.441760 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6841c9a-9b97-4ec5-bc02-372fb033b908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:13.442496 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6841c9a-9b97-4ec5-bc02-372fb033b908 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:13.442782 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6841c9a-9b97-4ec5-bc02-372fb033b908 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:13.443136 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6841c9a-9b97-4ec5-bc02-372fb033b908 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:13.444154 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6841c9a-9b97-4ec5-bc02-372fb033b908 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:13.446039 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6841c9a-9b97-4ec5-bc02-372fb033b908 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:13.521286 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1333/2686] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:13:13 2022] POST /identity/v3/projects => generated 401 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:13:13.525188 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1755a864-444b-455c-bf8f-1b5255647381 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:13.537138 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1755a864-444b-455c-bf8f-1b5255647381 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:13.601695 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5802ab6-6164-4fff-8cb6-554cf5226d66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:13.602396 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5802ab6-6164-4fff-8cb6-554cf5226d66 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:13.602714 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5802ab6-6164-4fff-8cb6-554cf5226d66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:13.602999 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5802ab6-6164-4fff-8cb6-554cf5226d66 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:13.603706 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5802ab6-6164-4fff-8cb6-554cf5226d66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:13.605784 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5802ab6-6164-4fff-8cb6-554cf5226d66 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:13.681977 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6841c9a-9b97-4ec5-bc02-372fb033b908 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:13.683790 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d6841c9a-9b97-4ec5-bc02-372fb033b908 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:13:13.693971 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1354/2687] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:13:13 2022] POST /identity/v3/users => generated 307 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 10 13:13:13.697911 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5802ab6-6164-4fff-8cb6-554cf5226d66 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:13.708422 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5802ab6-6164-4fff-8cb6-554cf5226d66 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:13.714632 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08441aca-a77e-45fa-9b3c-ffbb11889a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:13.715430 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08441aca-a77e-45fa-9b3c-ffbb11889a78 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:13.715778 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08441aca-a77e-45fa-9b3c-ffbb11889a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:13.716091 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08441aca-a77e-45fa-9b3c-ffbb11889a78 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:13.716817 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08441aca-a77e-45fa-9b3c-ffbb11889a78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:13.719005 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08441aca-a77e-45fa-9b3c-ffbb11889a78 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:13.734233 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:13:13.735964 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1355/2688] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:13:13 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 10 13:13:13.736900 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08441aca-a77e-45fa-9b3c-ffbb11889a78 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:13.757340 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6841c9a-9b97-4ec5-bc02-372fb033b908 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:13.757805 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08441aca-a77e-45fa-9b3c-ffbb11889a78 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:13.762706 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f7fad0-c774-4eec-9e10-27960cfc6914 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:13.762912 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298d74d3-36be-497f-9926-404e464a0374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:13.763368 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f7fad0-c774-4eec-9e10-27960cfc6914 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:13.763655 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f7fad0-c774-4eec-9e10-27960cfc6914 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:13.763914 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298d74d3-36be-497f-9926-404e464a0374 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:13.764047 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f7fad0-c774-4eec-9e10-27960cfc6914 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:13.764212 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298d74d3-36be-497f-9926-404e464a0374 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:13.764550 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298d74d3-36be-497f-9926-404e464a0374 admin admin] PATH_INFO: `/v3/projects/30aa8992c2324272be823c1022579a86/users/2c4ceec2d6fb4d5d93bd7dda9637be6b/roles/b43d5a8247e244869c2c70f986f42e8a` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:13.766367 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-36f7fad0-c774-4eec-9e10-27960cfc6914 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:13:13.780463 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f7fad0-c774-4eec-9e10-27960cfc6914 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:13.781924 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f7fad0-c774-4eec-9e10-27960cfc6914 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:13.786350 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298d74d3-36be-497f-9926-404e464a0374 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30aa8992c2324272be823c1022579a86, user_id=2c4ceec2d6fb4d5d93bd7dda9637be6b, role_id=b43d5a8247e244869c2c70f986f42e8a)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:13.789350 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298d74d3-36be-497f-9926-404e464a0374 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:13.794274 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1334/2689] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:13:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 10 13:13:13.860647 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1356/2690] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:13:13 2022] PUT /identity/v3/projects/30aa8992c2324272be823c1022579a86/users/2c4ceec2d6fb4d5d93bd7dda9637be6b/roles/b43d5a8247e244869c2c70f986f42e8a => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:13:13.864973 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f7fad0-c774-4eec-9e10-27960cfc6914 service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:13.876530 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f7fad0-c774-4eec-9e10-27960cfc6914 service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:13.933344 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc90fe0-66e5-426e-b34c-d2d6e5617a44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:13.934018 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc90fe0-66e5-426e-b34c-d2d6e5617a44 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:13.934301 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc90fe0-66e5-426e-b34c-d2d6e5617a44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:13.934576 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc90fe0-66e5-426e-b34c-d2d6e5617a44 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:13.935305 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc90fe0-66e5-426e-b34c-d2d6e5617a44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:13.937485 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc90fe0-66e5-426e-b34c-d2d6e5617a44 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:13.947587 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1335/2691] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:13:13 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:13:13.951726 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298d74d3-36be-497f-9926-404e464a0374 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:13.962563 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298d74d3-36be-497f-9926-404e464a0374 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:13.967285 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0a72af-85fe-47bf-9b06-64c220d8193c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:13.974217 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0a72af-85fe-47bf-9b06-64c220d8193c admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:13.974569 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0a72af-85fe-47bf-9b06-64c220d8193c admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:13.974796 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0a72af-85fe-47bf-9b06-64c220d8193c admin admin] PATH_INFO: `/v3/projects/30aa8992c2324272be823c1022579a86/users/2c4ceec2d6fb4d5d93bd7dda9637be6b/roles/b43d5a8247e244869c2c70f986f42e8a` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:13.976941 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0a72af-85fe-47bf-9b06-64c220d8193c admin admin] RBAC: Authorizing `identity:create_grant(project_id=30aa8992c2324272be823c1022579a86, user_id=2c4ceec2d6fb4d5d93bd7dda9637be6b, role_id=b43d5a8247e244869c2c70f986f42e8a)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:13.979652 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0a72af-85fe-47bf-9b06-64c220d8193c admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:13.992168 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1357/2692] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:13:13 2022] PUT /identity/v3/projects/30aa8992c2324272be823c1022579a86/users/2c4ceec2d6fb4d5d93bd7dda9637be6b/roles/b43d5a8247e244869c2c70f986f42e8a => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:13:14.003749 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d5178e-36b6-4d75-9829-dc106053b6dd None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:14.004175 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d5178e-36b6-4d75-9829-dc106053b6dd None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:14.004888 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d5178e-36b6-4d75-9829-dc106053b6dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:14.034102 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-95d5178e-36b6-4d75-9829-dc106053b6dd None None] MFA Rules not processed for user `2c4ceec2d6fb4d5d93bd7dda9637be6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:13:14.035427 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-95d5178e-36b6-4d75-9829-dc106053b6dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:13:14.106642 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1336/2693] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:13:14 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:13:14.117536 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0a72af-85fe-47bf-9b06-64c220d8193c admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:14.119272 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8b0a72af-85fe-47bf-9b06-64c220d8193c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:13:14.154084 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:13:14.179125 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0a72af-85fe-47bf-9b06-64c220d8193c admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:14.183404 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655fd44d-0e7a-415c-b91a-40f6d6a1e704 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6cc95a563b941ac92ab283aa0d5a5d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:14.184211 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655fd44d-0e7a-415c-b91a-40f6d6a1e704 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:14.184516 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655fd44d-0e7a-415c-b91a-40f6d6a1e704 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:14.184797 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655fd44d-0e7a-415c-b91a-40f6d6a1e704 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:14.194792 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655fd44d-0e7a-415c-b91a-40f6d6a1e704 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:14.196255 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655fd44d-0e7a-415c-b91a-40f6d6a1e704 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:14.206461 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1358/2694] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:13:14 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:13:14.395479 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d5178e-36b6-4d75-9829-dc106053b6dd None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:14.406469 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d5178e-36b6-4d75-9829-dc106053b6dd None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:14.410146 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fb6e12-f3cd-402e-b2b5-2317aafb1b6c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6cc95a563b941ac92ab283aa0d5a5d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:14.410798 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fb6e12-f3cd-402e-b2b5-2317aafb1b6c service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:14.411145 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fb6e12-f3cd-402e-b2b5-2317aafb1b6c service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:14.411427 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fb6e12-f3cd-402e-b2b5-2317aafb1b6c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:14.471560 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fb6e12-f3cd-402e-b2b5-2317aafb1b6c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:14.473098 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fb6e12-f3cd-402e-b2b5-2317aafb1b6c service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:14.476915 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=100 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:13:14.483172 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1337/2695] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:13:14 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:13:17.751557 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655fd44d-0e7a-415c-b91a-40f6d6a1e704 service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:17.754708 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-655fd44d-0e7a-415c-b91a-40f6d6a1e704 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:13:17.830508 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655fd44d-0e7a-415c-b91a-40f6d6a1e704 service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:17.833122 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7820fcc7-5563-4e99-a8dc-27ae9dc66b89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:17.833994 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7820fcc7-5563-4e99-a8dc-27ae9dc66b89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:17.834283 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7820fcc7-5563-4e99-a8dc-27ae9dc66b89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:17.834563 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7820fcc7-5563-4e99-a8dc-27ae9dc66b89 admin admin] PATH_INFO: `/v3/users/c71b903e00214894b94b03b3401607a0` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:17.835767 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7820fcc7-5563-4e99-a8dc-27ae9dc66b89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c71b903e00214894b94b03b3401607a0)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:17.837747 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7820fcc7-5563-4e99-a8dc-27ae9dc66b89 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:18.044664 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-7820fcc7-5563-4e99-a8dc-27ae9dc66b89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c71b903e00214894b94b03b3401607a0'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:18.052728 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-7820fcc7-5563-4e99-a8dc-27ae9dc66b89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c71b903e00214894b94b03b3401607a0'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:18.107185 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-7820fcc7-5563-4e99-a8dc-27ae9dc66b89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c71b903e00214894b94b03b3401607a0'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:18.107809 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-7820fcc7-5563-4e99-a8dc-27ae9dc66b89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c71b903e00214894b94b03b3401607a0'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:18.121217 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-7820fcc7-5563-4e99-a8dc-27ae9dc66b89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c71b903e00214894b94b03b3401607a0'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:18.127280 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1359/2696] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:13:17 2022] DELETE /identity/v3/users/c71b903e00214894b94b03b3401607a0 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:13:18.421783 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fb6e12-f3cd-402e-b2b5-2317aafb1b6c service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:18.433814 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fb6e12-f3cd-402e-b2b5-2317aafb1b6c service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:18.496163 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a24675-8648-48c7-a9f6-b3b1ac855377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:18.497178 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a24675-8648-48c7-a9f6-b3b1ac855377 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:18.497524 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a24675-8648-48c7-a9f6-b3b1ac855377 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:18.497865 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a24675-8648-48c7-a9f6-b3b1ac855377 admin admin] PATH_INFO: `/v3/projects/2ffa0e3d9fb04472ab9543a7737b87ae` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:18.513626 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a24675-8648-48c7-a9f6-b3b1ac855377 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ffa0e3d9fb04472ab9543a7737b87ae)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:18.516330 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a24675-8648-48c7-a9f6-b3b1ac855377 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:18.651071 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-41a24675-8648-48c7-a9f6-b3b1ac855377 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ffa0e3d9fb04472ab9543a7737b87ae', 'request_id': 'req-41a24675-8648-48c7-a9f6-b3b1ac855377'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:18.657413 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-41a24675-8648-48c7-a9f6-b3b1ac855377 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ffa0e3d9fb04472ab9543a7737b87ae', 'request_id': 'req-41a24675-8648-48c7-a9f6-b3b1ac855377'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:18.664202 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-41a24675-8648-48c7-a9f6-b3b1ac855377 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ffa0e3d9fb04472ab9543a7737b87ae', 'request_id': 'req-41a24675-8648-48c7-a9f6-b3b1ac855377'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:18.746870 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-41a24675-8648-48c7-a9f6-b3b1ac855377 admin admin] The token cache is being invalidate because project 2ffa0e3d9fb04472ab9543a7737b87ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:13:18.747161 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-41a24675-8648-48c7-a9f6-b3b1ac855377 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:18.747870 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-41a24675-8648-48c7-a9f6-b3b1ac855377 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:18.749425 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1338/2697] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:13:18 2022] DELETE /identity/v3/projects/2ffa0e3d9fb04472ab9543a7737b87ae => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:13:18.754443 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88e7ebb-f46c-41b8-8fd1-5c7678161cd9 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:18.754912 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88e7ebb-f46c-41b8-8fd1-5c7678161cd9 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:18.755013 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88e7ebb-f46c-41b8-8fd1-5c7678161cd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:18.773965 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-b88e7ebb-f46c-41b8-8fd1-5c7678161cd9 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:13:18.775177 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b88e7ebb-f46c-41b8-8fd1-5c7678161cd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:13:18.823537 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:13:18.836151 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1360/2698] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:13:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:13:18.845838 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5241cb-730b-4a2d-b9c8-37e8c3615771 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:18.846264 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5241cb-730b-4a2d-b9c8-37e8c3615771 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:18.848238 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5241cb-730b-4a2d-b9c8-37e8c3615771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:18.866805 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-1f5241cb-730b-4a2d-b9c8-37e8c3615771 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:13:18.868059 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-1f5241cb-730b-4a2d-b9c8-37e8c3615771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:13:18.876887 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1339/2699] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:13:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 10 13:13:18.881674 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88e7ebb-f46c-41b8-8fd1-5c7678161cd9 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:18.892373 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88e7ebb-f46c-41b8-8fd1-5c7678161cd9 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:18.897724 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1a5333-c4ad-4f30-865d-c48e50e77d30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:18.898392 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1a5333-c4ad-4f30-865d-c48e50e77d30 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:18.898680 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1a5333-c4ad-4f30-865d-c48e50e77d30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:18.898960 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1a5333-c4ad-4f30-865d-c48e50e77d30 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:18.899690 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1a5333-c4ad-4f30-865d-c48e50e77d30 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 13:13:18.900036 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1a5333-c4ad-4f30-865d-c48e50e77d30 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:18.901691 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1a5333-c4ad-4f30-865d-c48e50e77d30 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:18.913922 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1361/2700] 10.222.0.27 () {58 vars in 1152 bytes} [Fri Jun 10 13:13:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 13:13:18.918621 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5241cb-730b-4a2d-b9c8-37e8c3615771 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:18.928323 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5241cb-730b-4a2d-b9c8-37e8c3615771 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:18.933311 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddb01b2-4994-4e56-9cda-99b19358511c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:18.935497 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddb01b2-4994-4e56-9cda-99b19358511c admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:18.935497 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddb01b2-4994-4e56-9cda-99b19358511c admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:18.935497 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddb01b2-4994-4e56-9cda-99b19358511c admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:18.935497 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddb01b2-4994-4e56-9cda-99b19358511c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:18.937185 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddb01b2-4994-4e56-9cda-99b19358511c admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:19.046569 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1340/2701] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:13:18 2022] POST /identity/v3/projects => generated 381 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:13:19.050367 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1a5333-c4ad-4f30-865d-c48e50e77d30 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:19.060467 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1a5333-c4ad-4f30-865d-c48e50e77d30 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:19.120999 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e3a58d-29de-4fe9-8340-bbb67a150e3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:19.121711 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e3a58d-29de-4fe9-8340-bbb67a150e3b admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:19.121998 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e3a58d-29de-4fe9-8340-bbb67a150e3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:19.122300 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e3a58d-29de-4fe9-8340-bbb67a150e3b admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:19.122995 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e3a58d-29de-4fe9-8340-bbb67a150e3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:19.124979 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e3a58d-29de-4fe9-8340-bbb67a150e3b admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:19.228658 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1362/2702] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:13:19 2022] POST /identity/v3/users => generated 291 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:13:19.232519 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddb01b2-4994-4e56-9cda-99b19358511c admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:19.243265 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddb01b2-4994-4e56-9cda-99b19358511c admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:19.248031 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c1fffa-33ea-49fe-8b34-e2e61e4c57a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:19.248677 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c1fffa-33ea-49fe-8b34-e2e61e4c57a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:19.248946 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c1fffa-33ea-49fe-8b34-e2e61e4c57a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:19.249224 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c1fffa-33ea-49fe-8b34-e2e61e4c57a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:19.249947 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c1fffa-33ea-49fe-8b34-e2e61e4c57a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:19.252235 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c1fffa-33ea-49fe-8b34-e2e61e4c57a7 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:19.271103 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1341/2703] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:13:19 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:13:19.272162 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e3a58d-29de-4fe9-8340-bbb67a150e3b admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:19.295372 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e3a58d-29de-4fe9-8340-bbb67a150e3b admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:19.295372 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471ca9c6-fb77-4d77-be9b-eb85d064f033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:19.295372 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471ca9c6-fb77-4d77-be9b-eb85d064f033 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:19.295372 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471ca9c6-fb77-4d77-be9b-eb85d064f033 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:19.296201 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471ca9c6-fb77-4d77-be9b-eb85d064f033 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:19.296201 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471ca9c6-fb77-4d77-be9b-eb85d064f033 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:19.298129 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471ca9c6-fb77-4d77-be9b-eb85d064f033 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:19.311521 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1363/2704] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:13:19 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:13:19.330332 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c1fffa-33ea-49fe-8b34-e2e61e4c57a7 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:19.335366 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c1fffa-33ea-49fe-8b34-e2e61e4c57a7 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:19.342622 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acc2afd-1087-43e6-83ed-2f74c88a0a06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:19.347215 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc2afd-1087-43e6-83ed-2f74c88a0a06 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:19.347215 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc2afd-1087-43e6-83ed-2f74c88a0a06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:19.347215 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc2afd-1087-43e6-83ed-2f74c88a0a06 admin admin] PATH_INFO: `/v3/projects/fb89ac142a0c4fc39b767477c74c5388/users/aabc6c6b0dfa430eb9db00e7b753b89c/roles/43106c664e1b40de984c0c62701149c8` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:19.360176 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acc2afd-1087-43e6-83ed-2f74c88a0a06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb89ac142a0c4fc39b767477c74c5388, user_id=aabc6c6b0dfa430eb9db00e7b753b89c, role_id=43106c664e1b40de984c0c62701149c8)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:19.362863 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acc2afd-1087-43e6-83ed-2f74c88a0a06 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:19.454588 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471ca9c6-fb77-4d77-be9b-eb85d064f033 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:19.456340 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-471ca9c6-fb77-4d77-be9b-eb85d064f033 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:13:19.468897 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1342/2705] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:13:19 2022] PUT /identity/v3/projects/fb89ac142a0c4fc39b767477c74c5388/users/aabc6c6b0dfa430eb9db00e7b753b89c/roles/43106c664e1b40de984c0c62701149c8 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:13:19.472065 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471ca9c6-fb77-4d77-be9b-eb85d064f033 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:19.479564 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f73b515-12c7-4ddd-b6b0-9f3155da06ca None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:19.479838 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f73b515-12c7-4ddd-b6b0-9f3155da06ca None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:19.480109 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f73b515-12c7-4ddd-b6b0-9f3155da06ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:19.510082 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-6f73b515-12c7-4ddd-b6b0-9f3155da06ca None None] MFA Rules not processed for user `aabc6c6b0dfa430eb9db00e7b753b89c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:13:19.512203 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-6f73b515-12c7-4ddd-b6b0-9f3155da06ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:13:19.516388 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:13:19.534345 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4674f074-dd13-4831-98e1-4c6952b7c500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:19.538336 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4674f074-dd13-4831-98e1-4c6952b7c500 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:19.538585 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4674f074-dd13-4831-98e1-4c6952b7c500 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:19.538870 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4674f074-dd13-4831-98e1-4c6952b7c500 admin admin] PATH_INFO: `/v3/users/cf2aca344ebe4bdfbcf5a6ea6b53d97a` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:19.541219 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4674f074-dd13-4831-98e1-4c6952b7c500 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf2aca344ebe4bdfbcf5a6ea6b53d97a)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:19.543221 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4674f074-dd13-4831-98e1-4c6952b7c500 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:19.580434 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1343/2706] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:13:19 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 10 13:13:19.584766 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f73b515-12c7-4ddd-b6b0-9f3155da06ca None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:19.595374 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f73b515-12c7-4ddd-b6b0-9f3155da06ca None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:19.600021 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c7bdc1-83b9-4188-902e-d782a18003da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:19.603409 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c7bdc1-83b9-4188-902e-d782a18003da admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:19.603409 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c7bdc1-83b9-4188-902e-d782a18003da admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:19.603409 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c7bdc1-83b9-4188-902e-d782a18003da admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:19.603409 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c7bdc1-83b9-4188-902e-d782a18003da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:19.603754 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c7bdc1-83b9-4188-902e-d782a18003da admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:19.694689 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4674f074-dd13-4831-98e1-4c6952b7c500 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf2aca344ebe4bdfbcf5a6ea6b53d97a'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:19.704248 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4674f074-dd13-4831-98e1-4c6952b7c500 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf2aca344ebe4bdfbcf5a6ea6b53d97a'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:19.708368 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:13:19.712864 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1344/2707] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:13:19 2022] POST /identity/v3/projects => generated 379 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 10 13:13:19.714570 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c7bdc1-83b9-4188-902e-d782a18003da admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:19.781833 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c7bdc1-83b9-4188-902e-d782a18003da admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:19.786683 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae27be7-58e8-4465-8584-f7b40259b166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:19.787363 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae27be7-58e8-4465-8584-f7b40259b166 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:19.787644 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae27be7-58e8-4465-8584-f7b40259b166 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:19.787963 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae27be7-58e8-4465-8584-f7b40259b166 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:19.788652 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae27be7-58e8-4465-8584-f7b40259b166 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:19.790601 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae27be7-58e8-4465-8584-f7b40259b166 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:19.799425 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4674f074-dd13-4831-98e1-4c6952b7c500 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf2aca344ebe4bdfbcf5a6ea6b53d97a'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:19.800073 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4674f074-dd13-4831-98e1-4c6952b7c500 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf2aca344ebe4bdfbcf5a6ea6b53d97a'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:19.814953 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-4674f074-dd13-4831-98e1-4c6952b7c500 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf2aca344ebe4bdfbcf5a6ea6b53d97a'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:19.821937 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1364/2708] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:13:19 2022] DELETE /identity/v3/users/cf2aca344ebe4bdfbcf5a6ea6b53d97a => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 10 13:13:19.826337 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6065ff21-5970-4410-8bb8-5aa1d5ab029a None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:19.826619 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6065ff21-5970-4410-8bb8-5aa1d5ab029a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:19.826897 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6065ff21-5970-4410-8bb8-5aa1d5ab029a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:19.847329 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-6065ff21-5970-4410-8bb8-5aa1d5ab029a None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:13:19.848704 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6065ff21-5970-4410-8bb8-5aa1d5ab029a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:13:19.905383 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1345/2709] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:13:19 2022] POST /identity/v3/users => generated 296 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:13:19.907855 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae27be7-58e8-4465-8584-f7b40259b166 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:19.910122 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-fae27be7-58e8-4465-8584-f7b40259b166 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:13:19.914200 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1365/2710] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:13:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 10 13:13:19.921486 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6065ff21-5970-4410-8bb8-5aa1d5ab029a None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:19.924150 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6065ff21-5970-4410-8bb8-5aa1d5ab029a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:13:19.927447 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae27be7-58e8-4465-8584-f7b40259b166 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:19.932672 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d419f49-0804-4dc5-b9b4-2b66ab1a2478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:19.934034 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d419f49-0804-4dc5-b9b4-2b66ab1a2478 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:19.934034 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d419f49-0804-4dc5-b9b4-2b66ab1a2478 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:19.934034 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d419f49-0804-4dc5-b9b4-2b66ab1a2478 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:19.934728 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d419f49-0804-4dc5-b9b4-2b66ab1a2478 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:19.936378 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d419f49-0804-4dc5-b9b4-2b66ab1a2478 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:19.951889 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1346/2711] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:13:19 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:13:19.953779 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d419f49-0804-4dc5-b9b4-2b66ab1a2478 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:19.964714 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d419f49-0804-4dc5-b9b4-2b66ab1a2478 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:19.969912 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1a914b-af54-4d6c-9ba2-cc77010278bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:19.970793 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1a914b-af54-4d6c-9ba2-cc77010278bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:19.971076 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1a914b-af54-4d6c-9ba2-cc77010278bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:19.971351 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1a914b-af54-4d6c-9ba2-cc77010278bc admin admin] PATH_INFO: `/v3/projects/186c2f1ff5b145efb24d1b005e6bb0e7/users/84c6add9e9454183884a60e63e4a73ae/roles/b43d5a8247e244869c2c70f986f42e8a` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:19.987449 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1a914b-af54-4d6c-9ba2-cc77010278bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=186c2f1ff5b145efb24d1b005e6bb0e7, user_id=84c6add9e9454183884a60e63e4a73ae, role_id=b43d5a8247e244869c2c70f986f42e8a)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:19.987631 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6065ff21-5970-4410-8bb8-5aa1d5ab029a None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:19.990497 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1a914b-af54-4d6c-9ba2-cc77010278bc admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:19.993487 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ef1e46-a2cd-4bf5-add5-673161c533b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:19.994651 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ef1e46-a2cd-4bf5-add5-673161c533b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:19.994940 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ef1e46-a2cd-4bf5-add5-673161c533b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:19.995221 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ef1e46-a2cd-4bf5-add5-673161c533b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:20.005781 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ef1e46-a2cd-4bf5-add5-673161c533b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:20.008416 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ef1e46-a2cd-4bf5-add5-673161c533b6 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:20.022567 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1366/2712] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:13:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:13:20.069929 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1347/2713] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:13:19 2022] PUT /identity/v3/projects/186c2f1ff5b145efb24d1b005e6bb0e7/users/84c6add9e9454183884a60e63e4a73ae/roles/b43d5a8247e244869c2c70f986f42e8a => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 10 13:13:20.074021 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ef1e46-a2cd-4bf5-add5-673161c533b6 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:20.084381 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ef1e46-a2cd-4bf5-add5-673161c533b6 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:20.147623 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7916288-b4ee-4b5d-954b-44918c3b8649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:20.148327 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7916288-b4ee-4b5d-954b-44918c3b8649 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:20.148611 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7916288-b4ee-4b5d-954b-44918c3b8649 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:20.148888 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7916288-b4ee-4b5d-954b-44918c3b8649 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:20.149613 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7916288-b4ee-4b5d-954b-44918c3b8649 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:20.151192 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7916288-b4ee-4b5d-954b-44918c3b8649 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:20.165683 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1367/2714] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:13:20 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 10 13:13:20.168783 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1a914b-af54-4d6c-9ba2-cc77010278bc admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:20.177885 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1a914b-af54-4d6c-9ba2-cc77010278bc admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:20.182627 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e8ae33-97ba-4822-8782-421b99885dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:20.183502 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e8ae33-97ba-4822-8782-421b99885dca admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:20.183786 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e8ae33-97ba-4822-8782-421b99885dca admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:20.184093 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e8ae33-97ba-4822-8782-421b99885dca admin admin] PATH_INFO: `/v3/projects/186c2f1ff5b145efb24d1b005e6bb0e7/users/84c6add9e9454183884a60e63e4a73ae/roles/b43d5a8247e244869c2c70f986f42e8a` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:20.186252 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e8ae33-97ba-4822-8782-421b99885dca admin admin] RBAC: Authorizing `identity:create_grant(project_id=186c2f1ff5b145efb24d1b005e6bb0e7, user_id=84c6add9e9454183884a60e63e4a73ae, role_id=b43d5a8247e244869c2c70f986f42e8a)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:20.189244 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e8ae33-97ba-4822-8782-421b99885dca admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:20.207064 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1348/2715] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:13:20 2022] PUT /identity/v3/projects/186c2f1ff5b145efb24d1b005e6bb0e7/users/84c6add9e9454183884a60e63e4a73ae/roles/b43d5a8247e244869c2c70f986f42e8a => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:13:20.217495 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e72338-1fe3-4ba9-ab3e-ee4433c97cdb None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:20.217787 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e72338-1fe3-4ba9-ab3e-ee4433c97cdb None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:20.218068 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e72338-1fe3-4ba9-ab3e-ee4433c97cdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:20.250559 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-a5e72338-1fe3-4ba9-ab3e-ee4433c97cdb None None] MFA Rules not processed for user `84c6add9e9454183884a60e63e4a73ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:13:20.251806 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a5e72338-1fe3-4ba9-ab3e-ee4433c97cdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:13:20.257176 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:13:20.323668 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1368/2716] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:13:20 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 10 13:13:20.335128 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e8ae33-97ba-4822-8782-421b99885dca admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:20.336834 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b0e8ae33-97ba-4822-8782-421b99885dca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:13:20.367197 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:13:20.398968 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e8ae33-97ba-4822-8782-421b99885dca admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:20.402883 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f48dd20-cfa2-4dc3-a135-200f9741ff27 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6cc95a563b941ac92ab283aa0d5a5d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:20.403532 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f48dd20-cfa2-4dc3-a135-200f9741ff27 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:20.403822 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f48dd20-cfa2-4dc3-a135-200f9741ff27 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:20.404154 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f48dd20-cfa2-4dc3-a135-200f9741ff27 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:20.413834 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f48dd20-cfa2-4dc3-a135-200f9741ff27 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:20.415319 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f48dd20-cfa2-4dc3-a135-200f9741ff27 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:20.426071 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1349/2717] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:13:20 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:13:20.451917 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e72338-1fe3-4ba9-ab3e-ee4433c97cdb None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:20.454012 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a5e72338-1fe3-4ba9-ab3e-ee4433c97cdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:13:20.523332 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e72338-1fe3-4ba9-ab3e-ee4433c97cdb None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:20.528002 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5465fd-992e-4f29-93e6-0b13378c4401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:20.528864 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5465fd-992e-4f29-93e6-0b13378c4401 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:20.529174 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5465fd-992e-4f29-93e6-0b13378c4401 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:20.529427 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5465fd-992e-4f29-93e6-0b13378c4401 admin admin] PATH_INFO: `/v3/projects/b9eab1f46720410cab695b55afc2a1ba` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:20.531140 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5465fd-992e-4f29-93e6-0b13378c4401 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9eab1f46720410cab695b55afc2a1ba)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:20.533209 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5465fd-992e-4f29-93e6-0b13378c4401 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:20.630070 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-de5465fd-992e-4f29-93e6-0b13378c4401 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9eab1f46720410cab695b55afc2a1ba', 'request_id': 'req-de5465fd-992e-4f29-93e6-0b13378c4401'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:20.636655 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-de5465fd-992e-4f29-93e6-0b13378c4401 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9eab1f46720410cab695b55afc2a1ba', 'request_id': 'req-de5465fd-992e-4f29-93e6-0b13378c4401'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:20.645951 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-de5465fd-992e-4f29-93e6-0b13378c4401 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9eab1f46720410cab695b55afc2a1ba', 'request_id': 'req-de5465fd-992e-4f29-93e6-0b13378c4401'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:20.653883 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f48dd20-cfa2-4dc3-a135-200f9741ff27 service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:20.663173 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f48dd20-cfa2-4dc3-a135-200f9741ff27 service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:20.712273 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-de5465fd-992e-4f29-93e6-0b13378c4401 admin admin] The token cache is being invalidate because project b9eab1f46720410cab695b55afc2a1ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:13:20.712554 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-de5465fd-992e-4f29-93e6-0b13378c4401 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:20.713179 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-de5465fd-992e-4f29-93e6-0b13378c4401 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:20.714093 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7339cc0-8a70-469d-aea5-5900ce48b1a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6cc95a563b941ac92ab283aa0d5a5d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:20.714734 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7339cc0-8a70-469d-aea5-5900ce48b1a0 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:20.714858 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1369/2718] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:13:20 2022] DELETE /identity/v3/projects/b9eab1f46720410cab695b55afc2a1ba => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:13:20.716195 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7339cc0-8a70-469d-aea5-5900ce48b1a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:20.716574 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7339cc0-8a70-469d-aea5-5900ce48b1a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:20.718925 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b7339cc0-8a70-469d-aea5-5900ce48b1a0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:13:20.719216 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5465fd-992e-4f29-93e6-0b13378c4401 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:20.720634 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-de5465fd-992e-4f29-93e6-0b13378c4401 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:13:20.758774 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:13:20.785346 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7339cc0-8a70-469d-aea5-5900ce48b1a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:20.786926 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7339cc0-8a70-469d-aea5-5900ce48b1a0 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:20.788781 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5465fd-992e-4f29-93e6-0b13378c4401 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:20.794809 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da91e7b6-4e91-48e2-bc5c-c3dd218cd7c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:20.795715 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da91e7b6-4e91-48e2-bc5c-c3dd218cd7c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:20.796021 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da91e7b6-4e91-48e2-bc5c-c3dd218cd7c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:20.796295 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da91e7b6-4e91-48e2-bc5c-c3dd218cd7c1 admin admin] PATH_INFO: `/v3/users/2c4ceec2d6fb4d5d93bd7dda9637be6b` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:20.798132 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da91e7b6-4e91-48e2-bc5c-c3dd218cd7c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c4ceec2d6fb4d5d93bd7dda9637be6b)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:20.798295 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1350/2719] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:13:20 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 10 13:13:20.800180 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da91e7b6-4e91-48e2-bc5c-c3dd218cd7c1 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:20.990033 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-da91e7b6-4e91-48e2-bc5c-c3dd218cd7c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c4ceec2d6fb4d5d93bd7dda9637be6b'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:20.997141 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-da91e7b6-4e91-48e2-bc5c-c3dd218cd7c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c4ceec2d6fb4d5d93bd7dda9637be6b'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:21.090213 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-da91e7b6-4e91-48e2-bc5c-c3dd218cd7c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c4ceec2d6fb4d5d93bd7dda9637be6b'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:21.090892 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-da91e7b6-4e91-48e2-bc5c-c3dd218cd7c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c4ceec2d6fb4d5d93bd7dda9637be6b'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:21.103664 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-da91e7b6-4e91-48e2-bc5c-c3dd218cd7c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c4ceec2d6fb4d5d93bd7dda9637be6b'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:21.110159 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1370/2720] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:13:20 2022] DELETE /identity/v3/users/2c4ceec2d6fb4d5d93bd7dda9637be6b => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 10 13:13:21.526307 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7339cc0-8a70-469d-aea5-5900ce48b1a0 service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:21.538273 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7339cc0-8a70-469d-aea5-5900ce48b1a0 service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:21.601495 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0d6d33-4ca9-44f7-b1e3-d6c99f4e383f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:21.602279 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0d6d33-4ca9-44f7-b1e3-d6c99f4e383f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:21.602559 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0d6d33-4ca9-44f7-b1e3-d6c99f4e383f admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:21.602839 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0d6d33-4ca9-44f7-b1e3-d6c99f4e383f admin admin] PATH_INFO: `/v3/projects/30aa8992c2324272be823c1022579a86` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:21.604032 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0d6d33-4ca9-44f7-b1e3-d6c99f4e383f admin admin] RBAC: Authorizing `identity:delete_project(project_id=30aa8992c2324272be823c1022579a86)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:21.605911 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0d6d33-4ca9-44f7-b1e3-d6c99f4e383f admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:21.630771 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:13:21.725544 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ed0d6d33-4ca9-44f7-b1e3-d6c99f4e383f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30aa8992c2324272be823c1022579a86', 'request_id': 'req-ed0d6d33-4ca9-44f7-b1e3-d6c99f4e383f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:21.731028 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ed0d6d33-4ca9-44f7-b1e3-d6c99f4e383f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30aa8992c2324272be823c1022579a86', 'request_id': 'req-ed0d6d33-4ca9-44f7-b1e3-d6c99f4e383f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:21.737291 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ed0d6d33-4ca9-44f7-b1e3-d6c99f4e383f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30aa8992c2324272be823c1022579a86', 'request_id': 'req-ed0d6d33-4ca9-44f7-b1e3-d6c99f4e383f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:21.804056 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ed0d6d33-4ca9-44f7-b1e3-d6c99f4e383f admin admin] The token cache is being invalidate because project 30aa8992c2324272be823c1022579a86 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:13:21.804351 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ed0d6d33-4ca9-44f7-b1e3-d6c99f4e383f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:21.804951 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ed0d6d33-4ca9-44f7-b1e3-d6c99f4e383f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:21.806490 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1351/2721] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:13:21 2022] DELETE /identity/v3/projects/30aa8992c2324272be823c1022579a86 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:13:21.813153 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73fd6a1-a607-4594-8a1d-dece099bf8b4 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:21.813631 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73fd6a1-a607-4594-8a1d-dece099bf8b4 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:21.813820 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73fd6a1-a607-4594-8a1d-dece099bf8b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:21.833302 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-c73fd6a1-a607-4594-8a1d-dece099bf8b4 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:13:21.834526 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c73fd6a1-a607-4594-8a1d-dece099bf8b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:13:21.895281 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1371/2722] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:13:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:13:21.903088 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892de0a1-b4be-48b5-8500-f1d2b7de61be None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:21.903380 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892de0a1-b4be-48b5-8500-f1d2b7de61be None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:21.903659 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892de0a1-b4be-48b5-8500-f1d2b7de61be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:21.922690 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-892de0a1-b4be-48b5-8500-f1d2b7de61be None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:13:21.923946 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-892de0a1-b4be-48b5-8500-f1d2b7de61be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:13:21.932017 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1352/2723] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:13:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:13:21.936174 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73fd6a1-a607-4594-8a1d-dece099bf8b4 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:21.946421 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73fd6a1-a607-4594-8a1d-dece099bf8b4 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:21.951007 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eb64e0-c195-4321-9b46-9f6ef6dff515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:21.951662 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eb64e0-c195-4321-9b46-9f6ef6dff515 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:21.951987 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eb64e0-c195-4321-9b46-9f6ef6dff515 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:21.952271 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eb64e0-c195-4321-9b46-9f6ef6dff515 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:21.953003 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31eb64e0-c195-4321-9b46-9f6ef6dff515 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 13:13:21.953362 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31eb64e0-c195-4321-9b46-9f6ef6dff515 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:21.955435 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31eb64e0-c195-4321-9b46-9f6ef6dff515 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:21.967747 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1372/2724] 10.222.0.27 () {58 vars in 1152 bytes} [Fri Jun 10 13:13:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 13:13:21.972102 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892de0a1-b4be-48b5-8500-f1d2b7de61be None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:21.981250 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892de0a1-b4be-48b5-8500-f1d2b7de61be None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:21.985792 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba84db8-58d9-491d-bc54-4a420273be53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:21.986433 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba84db8-58d9-491d-bc54-4a420273be53 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:21.986713 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba84db8-58d9-491d-bc54-4a420273be53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:21.986986 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba84db8-58d9-491d-bc54-4a420273be53 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:21.987675 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba84db8-58d9-491d-bc54-4a420273be53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:21.989573 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba84db8-58d9-491d-bc54-4a420273be53 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:22.057341 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1353/2725] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:13:21 2022] POST /identity/v3/projects => generated 369 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:13:22.061659 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eb64e0-c195-4321-9b46-9f6ef6dff515 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:22.073079 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eb64e0-c195-4321-9b46-9f6ef6dff515 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:22.112782 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:13:22.131606 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db20d628-4408-4f3c-89c8-92f6d29bfe12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:22.132326 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db20d628-4408-4f3c-89c8-92f6d29bfe12 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:22.132613 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db20d628-4408-4f3c-89c8-92f6d29bfe12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:22.132892 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db20d628-4408-4f3c-89c8-92f6d29bfe12 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:22.133580 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db20d628-4408-4f3c-89c8-92f6d29bfe12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:22.135501 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db20d628-4408-4f3c-89c8-92f6d29bfe12 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:22.307812 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1373/2726] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:13:22 2022] POST /identity/v3/users => generated 285 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:13:22.311935 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba84db8-58d9-491d-bc54-4a420273be53 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:22.322638 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba84db8-58d9-491d-bc54-4a420273be53 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:22.327198 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ee7fde-df3e-466a-b3d1-d265770f32c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:22.327874 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ee7fde-df3e-466a-b3d1-d265770f32c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:22.328178 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ee7fde-df3e-466a-b3d1-d265770f32c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:22.328459 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ee7fde-df3e-466a-b3d1-d265770f32c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:22.329188 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ee7fde-df3e-466a-b3d1-d265770f32c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:22.330816 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ee7fde-df3e-466a-b3d1-d265770f32c6 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:22.345910 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1354/2727] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:13:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:13:22.351007 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db20d628-4408-4f3c-89c8-92f6d29bfe12 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:22.363477 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db20d628-4408-4f3c-89c8-92f6d29bfe12 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:22.368221 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b14121b-a31f-41d6-a698-010fb17ad716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:22.371297 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b14121b-a31f-41d6-a698-010fb17ad716 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:22.371297 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b14121b-a31f-41d6-a698-010fb17ad716 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:22.371297 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b14121b-a31f-41d6-a698-010fb17ad716 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:22.371297 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b14121b-a31f-41d6-a698-010fb17ad716 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:22.371964 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b14121b-a31f-41d6-a698-010fb17ad716 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:22.384140 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1374/2728] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:13:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:13:22.388278 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ee7fde-df3e-466a-b3d1-d265770f32c6 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:22.398215 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ee7fde-df3e-466a-b3d1-d265770f32c6 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:22.406448 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66efe53a-dceb-4fb5-abce-99b5fdf8445a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:22.406448 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66efe53a-dceb-4fb5-abce-99b5fdf8445a admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:22.406448 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66efe53a-dceb-4fb5-abce-99b5fdf8445a admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:22.406448 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66efe53a-dceb-4fb5-abce-99b5fdf8445a admin admin] PATH_INFO: `/v3/projects/46e74cb9018c4bd8904f84fc943f1a18/users/e35f796f95b342ab9e2f514e62a4e44b/roles/43106c664e1b40de984c0c62701149c8` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:22.420975 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66efe53a-dceb-4fb5-abce-99b5fdf8445a admin admin] RBAC: Authorizing `identity:create_grant(project_id=46e74cb9018c4bd8904f84fc943f1a18, user_id=e35f796f95b342ab9e2f514e62a4e44b, role_id=43106c664e1b40de984c0c62701149c8)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:22.424200 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66efe53a-dceb-4fb5-abce-99b5fdf8445a admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:22.501727 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1355/2729] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:13:22 2022] PUT /identity/v3/projects/46e74cb9018c4bd8904f84fc943f1a18/users/e35f796f95b342ab9e2f514e62a4e44b/roles/43106c664e1b40de984c0c62701149c8 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:13:22.512612 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff04daa-bb29-432a-872c-d422eef09e5c None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:22.512915 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff04daa-bb29-432a-872c-d422eef09e5c None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:22.513207 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff04daa-bb29-432a-872c-d422eef09e5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:22.540923 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-bff04daa-bb29-432a-872c-d422eef09e5c None None] MFA Rules not processed for user `e35f796f95b342ab9e2f514e62a4e44b`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:13:22.542149 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-bff04daa-bb29-432a-872c-d422eef09e5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:13:22.609927 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1375/2730] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:13:22 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:13:22.621094 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66efe53a-dceb-4fb5-abce-99b5fdf8445a admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:22.622760 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-66efe53a-dceb-4fb5-abce-99b5fdf8445a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:13:22.692161 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66efe53a-dceb-4fb5-abce-99b5fdf8445a admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:22.695887 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e77b140-c6df-48c7-ac53-d31a0d61e135 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d3eb01a98274050b930616b98998c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:22.696558 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e77b140-c6df-48c7-ac53-d31a0d61e135 service glance] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:22.696830 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e77b140-c6df-48c7-ac53-d31a0d61e135 service glance] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:22.697111 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e77b140-c6df-48c7-ac53-d31a0d61e135 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:22.705942 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e77b140-c6df-48c7-ac53-d31a0d61e135 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:22.707450 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e77b140-c6df-48c7-ac53-d31a0d61e135 service glance] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:22.717166 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1356/2731] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:13:22 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:13:29.370076 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff04daa-bb29-432a-872c-d422eef09e5c None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:29.371469 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-bff04daa-bb29-432a-872c-d422eef09e5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:13:29.443993 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff04daa-bb29-432a-872c-d422eef09e5c None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:29.448561 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57011481-d7aa-4b18-908a-9a8601a183f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:29.449426 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57011481-d7aa-4b18-908a-9a8601a183f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:29.449707 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57011481-d7aa-4b18-908a-9a8601a183f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:29.449982 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57011481-d7aa-4b18-908a-9a8601a183f5 admin admin] PATH_INFO: `/v3/users/aabc6c6b0dfa430eb9db00e7b753b89c` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:29.451152 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57011481-d7aa-4b18-908a-9a8601a183f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aabc6c6b0dfa430eb9db00e7b753b89c)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:29.453077 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57011481-d7aa-4b18-908a-9a8601a183f5 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:29.647293 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-57011481-d7aa-4b18-908a-9a8601a183f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aabc6c6b0dfa430eb9db00e7b753b89c'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:29.649333 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:13:29.654387 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-57011481-d7aa-4b18-908a-9a8601a183f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aabc6c6b0dfa430eb9db00e7b753b89c'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:29.746994 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-57011481-d7aa-4b18-908a-9a8601a183f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aabc6c6b0dfa430eb9db00e7b753b89c'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:29.747613 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-57011481-d7aa-4b18-908a-9a8601a183f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aabc6c6b0dfa430eb9db00e7b753b89c'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:29.763177 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-57011481-d7aa-4b18-908a-9a8601a183f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aabc6c6b0dfa430eb9db00e7b753b89c'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:29.770048 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1376/2732] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:13:29 2022] DELETE /identity/v3/users/aabc6c6b0dfa430eb9db00e7b753b89c => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:13:29.775375 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22e9420-43ef-40c9-a325-a57dad36180c None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:29.775946 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22e9420-43ef-40c9-a325-a57dad36180c None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:29.776059 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22e9420-43ef-40c9-a325-a57dad36180c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:29.794439 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-a22e9420-43ef-40c9-a325-a57dad36180c None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:13:29.795653 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a22e9420-43ef-40c9-a325-a57dad36180c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:13:29.844472 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=91,keystone:DELETE=6,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:13:29.868794 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1357/2733] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:13:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 10 13:13:29.877769 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57011481-d7aa-4b18-908a-9a8601a183f5 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:29.879222 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-57011481-d7aa-4b18-908a-9a8601a183f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:13:29.940676 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57011481-d7aa-4b18-908a-9a8601a183f5 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:29.945011 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cd71c7-9a93-4b18-b3fc-539f9194962b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:29.945646 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cd71c7-9a93-4b18-b3fc-539f9194962b service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:29.945929 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cd71c7-9a93-4b18-b3fc-539f9194962b service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:29.946204 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cd71c7-9a93-4b18-b3fc-539f9194962b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:29.954866 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cd71c7-9a93-4b18-b3fc-539f9194962b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:29.956321 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cd71c7-9a93-4b18-b3fc-539f9194962b service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:29.965518 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1377/2734] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:13:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:13:30.394800 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22e9420-43ef-40c9-a325-a57dad36180c None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:30.396775 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a22e9420-43ef-40c9-a325-a57dad36180c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:13:30.409393 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22e9420-43ef-40c9-a325-a57dad36180c None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:30.414583 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d8970c-f107-4c6b-8b33-fc465ba713b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:30.418248 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d8970c-f107-4c6b-8b33-fc465ba713b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:30.418248 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d8970c-f107-4c6b-8b33-fc465ba713b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:30.418248 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d8970c-f107-4c6b-8b33-fc465ba713b1 admin admin] PATH_INFO: `/v3/projects/fb89ac142a0c4fc39b767477c74c5388` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:30.418248 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d8970c-f107-4c6b-8b33-fc465ba713b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb89ac142a0c4fc39b767477c74c5388)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:30.419091 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d8970c-f107-4c6b-8b33-fc465ba713b1 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:30.506911 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-43d8970c-f107-4c6b-8b33-fc465ba713b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb89ac142a0c4fc39b767477c74c5388', 'request_id': 'req-43d8970c-f107-4c6b-8b33-fc465ba713b1'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:30.512817 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-43d8970c-f107-4c6b-8b33-fc465ba713b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb89ac142a0c4fc39b767477c74c5388', 'request_id': 'req-43d8970c-f107-4c6b-8b33-fc465ba713b1'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:30.519011 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-43d8970c-f107-4c6b-8b33-fc465ba713b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb89ac142a0c4fc39b767477c74c5388', 'request_id': 'req-43d8970c-f107-4c6b-8b33-fc465ba713b1'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:30.568886 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-43d8970c-f107-4c6b-8b33-fc465ba713b1 admin admin] The token cache is being invalidate because project fb89ac142a0c4fc39b767477c74c5388 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:13:30.569565 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-43d8970c-f107-4c6b-8b33-fc465ba713b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:30.570067 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-43d8970c-f107-4c6b-8b33-fc465ba713b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:30.571611 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1358/2735] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:13:30 2022] DELETE /identity/v3/projects/fb89ac142a0c4fc39b767477c74c5388 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:13:30.575685 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cd71c7-9a93-4b18-b3fc-539f9194962b service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:30.586284 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cd71c7-9a93-4b18-b3fc-539f9194962b service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:30.645368 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6b61be-009e-49c2-b1b4-fb3e6665448a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:30.647083 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6b61be-009e-49c2-b1b4-fb3e6665448a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:30.647462 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6b61be-009e-49c2-b1b4-fb3e6665448a admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:30.647789 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6b61be-009e-49c2-b1b4-fb3e6665448a admin admin] PATH_INFO: `/v3/users/84c6add9e9454183884a60e63e4a73ae` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:30.649181 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6b61be-009e-49c2-b1b4-fb3e6665448a admin admin] RBAC: Authorizing `identity:delete_user(user_id=84c6add9e9454183884a60e63e4a73ae)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:30.651109 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6b61be-009e-49c2-b1b4-fb3e6665448a admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:30.852769 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6e6b61be-009e-49c2-b1b4-fb3e6665448a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84c6add9e9454183884a60e63e4a73ae'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:30.860025 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6e6b61be-009e-49c2-b1b4-fb3e6665448a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84c6add9e9454183884a60e63e4a73ae'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:30.935844 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6e6b61be-009e-49c2-b1b4-fb3e6665448a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84c6add9e9454183884a60e63e4a73ae'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:30.936511 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6e6b61be-009e-49c2-b1b4-fb3e6665448a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84c6add9e9454183884a60e63e4a73ae'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:30.944495 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:13:30.949906 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6e6b61be-009e-49c2-b1b4-fb3e6665448a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84c6add9e9454183884a60e63e4a73ae'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:30.955729 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1378/2736] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:13:30 2022] DELETE /identity/v3/users/84c6add9e9454183884a60e63e4a73ae => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 10 13:13:31.411233 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d8970c-f107-4c6b-8b33-fc465ba713b1 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:31.413284 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-43d8970c-f107-4c6b-8b33-fc465ba713b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:13:31.426813 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d8970c-f107-4c6b-8b33-fc465ba713b1 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:31.483617 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2e14f9-990a-4ff6-8140-54bba0d76d10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:31.484692 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2e14f9-990a-4ff6-8140-54bba0d76d10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:31.484974 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2e14f9-990a-4ff6-8140-54bba0d76d10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:31.485251 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2e14f9-990a-4ff6-8140-54bba0d76d10 admin admin] PATH_INFO: `/v3/projects/186c2f1ff5b145efb24d1b005e6bb0e7` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:31.486521 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2e14f9-990a-4ff6-8140-54bba0d76d10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=186c2f1ff5b145efb24d1b005e6bb0e7)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:31.488524 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2e14f9-990a-4ff6-8140-54bba0d76d10 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:31.577982 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ec2e14f9-990a-4ff6-8140-54bba0d76d10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '186c2f1ff5b145efb24d1b005e6bb0e7', 'request_id': 'req-ec2e14f9-990a-4ff6-8140-54bba0d76d10'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:31.584001 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ec2e14f9-990a-4ff6-8140-54bba0d76d10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '186c2f1ff5b145efb24d1b005e6bb0e7', 'request_id': 'req-ec2e14f9-990a-4ff6-8140-54bba0d76d10'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:31.590195 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ec2e14f9-990a-4ff6-8140-54bba0d76d10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '186c2f1ff5b145efb24d1b005e6bb0e7', 'request_id': 'req-ec2e14f9-990a-4ff6-8140-54bba0d76d10'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:31.642912 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ec2e14f9-990a-4ff6-8140-54bba0d76d10 admin admin] The token cache is being invalidate because project 186c2f1ff5b145efb24d1b005e6bb0e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:13:31.643199 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ec2e14f9-990a-4ff6-8140-54bba0d76d10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:31.643817 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ec2e14f9-990a-4ff6-8140-54bba0d76d10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:31.646136 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1359/2737] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:13:31 2022] DELETE /identity/v3/projects/186c2f1ff5b145efb24d1b005e6bb0e7 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:13:31.651209 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7825973-cf35-4697-8786-b2ceceadc210 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:31.651744 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7825973-cf35-4697-8786-b2ceceadc210 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:31.652022 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7825973-cf35-4697-8786-b2ceceadc210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:31.671889 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-c7825973-cf35-4697-8786-b2ceceadc210 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:13:31.673060 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c7825973-cf35-4697-8786-b2ceceadc210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:13:31.732798 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1379/2738] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:13:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 10 13:13:31.742665 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98258142-a6ad-41fa-a46e-15258c5a6eef None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:31.743025 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98258142-a6ad-41fa-a46e-15258c5a6eef None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:31.743249 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98258142-a6ad-41fa-a46e-15258c5a6eef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:31.750788 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:13:31.762051 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-98258142-a6ad-41fa-a46e-15258c5a6eef None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:13:31.766340 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-98258142-a6ad-41fa-a46e-15258c5a6eef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:13:31.771232 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1360/2739] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:13:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:13:31.775502 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7825973-cf35-4697-8786-b2ceceadc210 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:31.785643 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7825973-cf35-4697-8786-b2ceceadc210 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:31.790784 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c6a17c-852b-4361-be31-3139a5df34b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:31.791438 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c6a17c-852b-4361-be31-3139a5df34b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:31.791756 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c6a17c-852b-4361-be31-3139a5df34b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:31.792066 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c6a17c-852b-4361-be31-3139a5df34b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:31.792825 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c6a17c-852b-4361-be31-3139a5df34b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 13:13:31.793133 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c6a17c-852b-4361-be31-3139a5df34b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:31.794722 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c6a17c-852b-4361-be31-3139a5df34b7 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:31.806864 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1380/2740] 10.222.0.27 () {58 vars in 1152 bytes} [Fri Jun 10 13:13:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 13:13:31.811087 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98258142-a6ad-41fa-a46e-15258c5a6eef None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:31.820293 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98258142-a6ad-41fa-a46e-15258c5a6eef None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:31.824985 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818b0c09-5b28-43df-9406-1d6ee0dd4960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:31.825671 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818b0c09-5b28-43df-9406-1d6ee0dd4960 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:31.826173 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818b0c09-5b28-43df-9406-1d6ee0dd4960 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:31.826742 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818b0c09-5b28-43df-9406-1d6ee0dd4960 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:31.827725 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818b0c09-5b28-43df-9406-1d6ee0dd4960 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:31.829800 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818b0c09-5b28-43df-9406-1d6ee0dd4960 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:31.912614 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1361/2741] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:13:31 2022] POST /identity/v3/projects => generated 365 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:13:31.913876 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c6a17c-852b-4361-be31-3139a5df34b7 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:31.927896 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c6a17c-852b-4361-be31-3139a5df34b7 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:31.993160 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5815e83-3697-466f-970e-4dbdcf768ae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:31.993841 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5815e83-3697-466f-970e-4dbdcf768ae0 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:31.994130 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5815e83-3697-466f-970e-4dbdcf768ae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:31.994408 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5815e83-3697-466f-970e-4dbdcf768ae0 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:31.995135 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5815e83-3697-466f-970e-4dbdcf768ae0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:31.997114 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5815e83-3697-466f-970e-4dbdcf768ae0 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:32.050327 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1381/2742] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:13:31 2022] POST /identity/v3/users => generated 283 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:13:32.055087 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818b0c09-5b28-43df-9406-1d6ee0dd4960 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:32.065316 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818b0c09-5b28-43df-9406-1d6ee0dd4960 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:32.070617 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07724d57-86a0-44cc-942f-625be6a427bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:32.071283 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07724d57-86a0-44cc-942f-625be6a427bf admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:32.071565 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07724d57-86a0-44cc-942f-625be6a427bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:32.071846 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07724d57-86a0-44cc-942f-625be6a427bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:32.072605 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07724d57-86a0-44cc-942f-625be6a427bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:32.074200 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07724d57-86a0-44cc-942f-625be6a427bf admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:32.084565 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1362/2743] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:13:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:13:32.088689 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5815e83-3697-466f-970e-4dbdcf768ae0 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:32.104898 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5815e83-3697-466f-970e-4dbdcf768ae0 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:32.109871 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746dde8e-279b-45dd-9bbf-1f1e6fa2da52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:32.110591 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746dde8e-279b-45dd-9bbf-1f1e6fa2da52 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:32.110880 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746dde8e-279b-45dd-9bbf-1f1e6fa2da52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:32.111162 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746dde8e-279b-45dd-9bbf-1f1e6fa2da52 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:32.111922 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746dde8e-279b-45dd-9bbf-1f1e6fa2da52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:32.113560 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746dde8e-279b-45dd-9bbf-1f1e6fa2da52 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:32.124419 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1382/2744] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:13:32 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:13:32.128626 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07724d57-86a0-44cc-942f-625be6a427bf admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:32.139314 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07724d57-86a0-44cc-942f-625be6a427bf admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:32.143918 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e75436-3339-44fc-9bae-35c0e90894a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:32.148375 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e75436-3339-44fc-9bae-35c0e90894a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:32.148375 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e75436-3339-44fc-9bae-35c0e90894a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:32.148375 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e75436-3339-44fc-9bae-35c0e90894a0 admin admin] PATH_INFO: `/v3/projects/613ceaf3719d459e956654325519a934/users/8d77f1026f314995ba2fe39e8bf68249/roles/43106c664e1b40de984c0c62701149c8` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:32.160283 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e75436-3339-44fc-9bae-35c0e90894a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=613ceaf3719d459e956654325519a934, user_id=8d77f1026f314995ba2fe39e8bf68249, role_id=43106c664e1b40de984c0c62701149c8)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:32.162959 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e75436-3339-44fc-9bae-35c0e90894a0 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:32.233307 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1363/2745] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:13:32 2022] PUT /identity/v3/projects/613ceaf3719d459e956654325519a934/users/8d77f1026f314995ba2fe39e8bf68249/roles/43106c664e1b40de984c0c62701149c8 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:13:32.243461 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a00482-18e6-4674-95e7-bad1da04c9d0 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:32.243752 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a00482-18e6-4674-95e7-bad1da04c9d0 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:32.244060 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a00482-18e6-4674-95e7-bad1da04c9d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:32.273768 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-93a00482-18e6-4674-95e7-bad1da04c9d0 None None] MFA Rules not processed for user `8d77f1026f314995ba2fe39e8bf68249`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:13:32.275020 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-93a00482-18e6-4674-95e7-bad1da04c9d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:13:32.319063 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:13:32.339460 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1383/2746] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:13:32 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:13:32.343951 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e75436-3339-44fc-9bae-35c0e90894a0 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:32.407228 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e75436-3339-44fc-9bae-35c0e90894a0 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:32.412149 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af991a5a-e184-4d50-8b4e-b9cf27e7a84f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:32.412805 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af991a5a-e184-4d50-8b4e-b9cf27e7a84f admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:32.413087 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af991a5a-e184-4d50-8b4e-b9cf27e7a84f admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:32.413363 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af991a5a-e184-4d50-8b4e-b9cf27e7a84f admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:32.414115 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af991a5a-e184-4d50-8b4e-b9cf27e7a84f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:32.416099 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af991a5a-e184-4d50-8b4e-b9cf27e7a84f admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:32.485677 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1364/2747] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:13:32 2022] POST /identity/v3/projects => generated 365 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:13:32.489649 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a00482-18e6-4674-95e7-bad1da04c9d0 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:32.500683 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a00482-18e6-4674-95e7-bad1da04c9d0 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:32.557878 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b6e23f-67cb-44ec-aa27-9c3e75bb6e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:32.558565 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b6e23f-67cb-44ec-aa27-9c3e75bb6e41 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:32.558856 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b6e23f-67cb-44ec-aa27-9c3e75bb6e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:32.559139 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b6e23f-67cb-44ec-aa27-9c3e75bb6e41 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:32.559851 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b6e23f-67cb-44ec-aa27-9c3e75bb6e41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:32.561796 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b6e23f-67cb-44ec-aa27-9c3e75bb6e41 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:32.650798 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1384/2748] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:13:32 2022] POST /identity/v3/users => generated 289 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:13:32.654702 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af991a5a-e184-4d50-8b4e-b9cf27e7a84f admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:32.666032 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af991a5a-e184-4d50-8b4e-b9cf27e7a84f admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:32.670549 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b8b7e4-a78a-49b0-82c9-9d24a2f625b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:32.671197 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b8b7e4-a78a-49b0-82c9-9d24a2f625b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:32.671481 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b8b7e4-a78a-49b0-82c9-9d24a2f625b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:32.671752 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b8b7e4-a78a-49b0-82c9-9d24a2f625b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:32.672507 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b8b7e4-a78a-49b0-82c9-9d24a2f625b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:32.674086 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b8b7e4-a78a-49b0-82c9-9d24a2f625b4 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:32.685527 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1365/2749] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:13:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:13:32.689575 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b6e23f-67cb-44ec-aa27-9c3e75bb6e41 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:32.699778 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b6e23f-67cb-44ec-aa27-9c3e75bb6e41 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:32.704412 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f25b459-1a6e-4d2e-a52b-0e18eb65157f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:32.705288 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f25b459-1a6e-4d2e-a52b-0e18eb65157f admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:32.705575 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f25b459-1a6e-4d2e-a52b-0e18eb65157f admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:32.705855 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f25b459-1a6e-4d2e-a52b-0e18eb65157f admin admin] PATH_INFO: `/v3/projects/01b21d696a4741fa9ba479157a1dfa5e/users/8c8c9adead804159ae6b3c6613412f70/roles/b43d5a8247e244869c2c70f986f42e8a` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:32.720811 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f25b459-1a6e-4d2e-a52b-0e18eb65157f admin admin] RBAC: Authorizing `identity:create_grant(project_id=01b21d696a4741fa9ba479157a1dfa5e, user_id=8c8c9adead804159ae6b3c6613412f70, role_id=b43d5a8247e244869c2c70f986f42e8a)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:32.723495 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f25b459-1a6e-4d2e-a52b-0e18eb65157f admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:32.811238 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1385/2750] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:13:32 2022] PUT /identity/v3/projects/01b21d696a4741fa9ba479157a1dfa5e/users/8c8c9adead804159ae6b3c6613412f70/roles/b43d5a8247e244869c2c70f986f42e8a => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 10 13:13:32.815195 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b8b7e4-a78a-49b0-82c9-9d24a2f625b4 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:32.825201 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b8b7e4-a78a-49b0-82c9-9d24a2f625b4 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:32.882120 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60a50bf-ed4f-41f4-b64d-4a6eb9e65d36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:32.882796 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60a50bf-ed4f-41f4-b64d-4a6eb9e65d36 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:32.883081 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60a50bf-ed4f-41f4-b64d-4a6eb9e65d36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:32.883385 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60a50bf-ed4f-41f4-b64d-4a6eb9e65d36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:32.884153 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60a50bf-ed4f-41f4-b64d-4a6eb9e65d36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:32.885786 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60a50bf-ed4f-41f4-b64d-4a6eb9e65d36 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:32.895625 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1366/2751] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:13:32 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:13:32.900337 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f25b459-1a6e-4d2e-a52b-0e18eb65157f admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:32.911362 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f25b459-1a6e-4d2e-a52b-0e18eb65157f admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:32.916076 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a7a675-85d5-4603-b02c-3f2185b8f369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:32.916966 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a7a675-85d5-4603-b02c-3f2185b8f369 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:32.917276 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a7a675-85d5-4603-b02c-3f2185b8f369 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:32.917543 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a7a675-85d5-4603-b02c-3f2185b8f369 admin admin] PATH_INFO: `/v3/projects/01b21d696a4741fa9ba479157a1dfa5e/users/8c8c9adead804159ae6b3c6613412f70/roles/b43d5a8247e244869c2c70f986f42e8a` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:32.919501 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a7a675-85d5-4603-b02c-3f2185b8f369 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01b21d696a4741fa9ba479157a1dfa5e, user_id=8c8c9adead804159ae6b3c6613412f70, role_id=b43d5a8247e244869c2c70f986f42e8a)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:32.922162 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a7a675-85d5-4603-b02c-3f2185b8f369 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:32.933233 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1386/2752] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:13:32 2022] PUT /identity/v3/projects/01b21d696a4741fa9ba479157a1dfa5e/users/8c8c9adead804159ae6b3c6613412f70/roles/b43d5a8247e244869c2c70f986f42e8a => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:13:32.944627 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5861d654-f954-4147-bc71-bfc83fe0e2b9 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:32.944921 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5861d654-f954-4147-bc71-bfc83fe0e2b9 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:32.945204 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5861d654-f954-4147-bc71-bfc83fe0e2b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:32.966944 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:13:32.975120 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-5861d654-f954-4147-bc71-bfc83fe0e2b9 None None] MFA Rules not processed for user `8c8c9adead804159ae6b3c6613412f70`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:13:32.976386 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5861d654-f954-4147-bc71-bfc83fe0e2b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:13:33.061102 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1367/2753] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:13:32 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:13:33.072282 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a7a675-85d5-4603-b02c-3f2185b8f369 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:33.073989 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-69a7a675-85d5-4603-b02c-3f2185b8f369 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:13:33.132269 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a7a675-85d5-4603-b02c-3f2185b8f369 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:33.135960 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f7c326-fb3b-4918-95a9-8b3006715eab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6cc95a563b941ac92ab283aa0d5a5d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:33.136598 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f7c326-fb3b-4918-95a9-8b3006715eab service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:33.136884 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f7c326-fb3b-4918-95a9-8b3006715eab service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:33.137164 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f7c326-fb3b-4918-95a9-8b3006715eab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:33.146096 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f7c326-fb3b-4918-95a9-8b3006715eab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:33.147522 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f7c326-fb3b-4918-95a9-8b3006715eab service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:33.157277 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1387/2754] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:13:33 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:13:33.377974 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5861d654-f954-4147-bc71-bfc83fe0e2b9 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:33.388254 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5861d654-f954-4147-bc71-bfc83fe0e2b9 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:33.391720 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb0d459-820a-42f2-a69b-203164aa76f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6cc95a563b941ac92ab283aa0d5a5d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:33.392383 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb0d459-820a-42f2-a69b-203164aa76f6 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:33.392662 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb0d459-820a-42f2-a69b-203164aa76f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:33.392936 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb0d459-820a-42f2-a69b-203164aa76f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:33.451279 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb0d459-820a-42f2-a69b-203164aa76f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:33.452761 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb0d459-820a-42f2-a69b-203164aa76f6 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:33.461893 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1368/2755] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:13:33 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:13:36.903913 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f7c326-fb3b-4918-95a9-8b3006715eab service cinder] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:36.945711 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:13:36.964411 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f7c326-fb3b-4918-95a9-8b3006715eab service cinder] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:36.969259 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d66233-3c30-484f-a69a-9e46e9b6b404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:36.970159 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d66233-3c30-484f-a69a-9e46e9b6b404 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:36.970644 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d66233-3c30-484f-a69a-9e46e9b6b404 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:36.971336 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d66233-3c30-484f-a69a-9e46e9b6b404 admin admin] PATH_INFO: `/v3/users/8d77f1026f314995ba2fe39e8bf68249` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:36.972549 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d66233-3c30-484f-a69a-9e46e9b6b404 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d77f1026f314995ba2fe39e8bf68249)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:36.974986 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d66233-3c30-484f-a69a-9e46e9b6b404 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:37.155278 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-57d66233-3c30-484f-a69a-9e46e9b6b404 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d77f1026f314995ba2fe39e8bf68249'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:37.163183 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-57d66233-3c30-484f-a69a-9e46e9b6b404 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d77f1026f314995ba2fe39e8bf68249'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:37.221421 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-57d66233-3c30-484f-a69a-9e46e9b6b404 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d77f1026f314995ba2fe39e8bf68249'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:37.222038 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-57d66233-3c30-484f-a69a-9e46e9b6b404 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d77f1026f314995ba2fe39e8bf68249'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:37.234688 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-57d66233-3c30-484f-a69a-9e46e9b6b404 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d77f1026f314995ba2fe39e8bf68249'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:37.240668 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1388/2756] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:13:36 2022] DELETE /identity/v3/users/8d77f1026f314995ba2fe39e8bf68249 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:13:37.245215 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e38385f-1226-4217-9fa0-576efb6e6137 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:37.245689 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e38385f-1226-4217-9fa0-576efb6e6137 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:37.245822 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e38385f-1226-4217-9fa0-576efb6e6137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:37.262718 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-4e38385f-1226-4217-9fa0-576efb6e6137 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:13:37.263933 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-4e38385f-1226-4217-9fa0-576efb6e6137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:13:37.322923 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1369/2757] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:13:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 10 13:13:37.332605 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d66233-3c30-484f-a69a-9e46e9b6b404 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:37.334575 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-57d66233-3c30-484f-a69a-9e46e9b6b404 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:13:37.390187 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d66233-3c30-484f-a69a-9e46e9b6b404 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:37.393804 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31cd97f-4ea4-4869-aeb1-35c51531557a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:37.394442 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31cd97f-4ea4-4869-aeb1-35c51531557a service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:37.394729 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31cd97f-4ea4-4869-aeb1-35c51531557a service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:37.395009 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31cd97f-4ea4-4869-aeb1-35c51531557a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:37.404414 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31cd97f-4ea4-4869-aeb1-35c51531557a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:37.406085 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31cd97f-4ea4-4869-aeb1-35c51531557a service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:37.415993 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1389/2758] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:13:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:13:37.856694 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e38385f-1226-4217-9fa0-576efb6e6137 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:37.858235 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-4e38385f-1226-4217-9fa0-576efb6e6137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:13:37.871251 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e38385f-1226-4217-9fa0-576efb6e6137 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:37.876649 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464c2a7c-309c-4455-8fbe-57a5594bbb0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:37.877584 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464c2a7c-309c-4455-8fbe-57a5594bbb0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:37.877867 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464c2a7c-309c-4455-8fbe-57a5594bbb0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:37.878143 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464c2a7c-309c-4455-8fbe-57a5594bbb0c admin admin] PATH_INFO: `/v3/projects/613ceaf3719d459e956654325519a934` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:37.879340 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464c2a7c-309c-4455-8fbe-57a5594bbb0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=613ceaf3719d459e956654325519a934)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:37.881967 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464c2a7c-309c-4455-8fbe-57a5594bbb0c admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:37.968654 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:13:37.980989 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-464c2a7c-309c-4455-8fbe-57a5594bbb0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '613ceaf3719d459e956654325519a934', 'request_id': 'req-464c2a7c-309c-4455-8fbe-57a5594bbb0c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:37.986292 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-464c2a7c-309c-4455-8fbe-57a5594bbb0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '613ceaf3719d459e956654325519a934', 'request_id': 'req-464c2a7c-309c-4455-8fbe-57a5594bbb0c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:37.992978 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-464c2a7c-309c-4455-8fbe-57a5594bbb0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '613ceaf3719d459e956654325519a934', 'request_id': 'req-464c2a7c-309c-4455-8fbe-57a5594bbb0c'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:38.077310 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-464c2a7c-309c-4455-8fbe-57a5594bbb0c admin admin] The token cache is being invalidate because project 613ceaf3719d459e956654325519a934 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:13:38.077599 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-464c2a7c-309c-4455-8fbe-57a5594bbb0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:38.078232 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-464c2a7c-309c-4455-8fbe-57a5594bbb0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:38.080080 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1370/2759] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:13:37 2022] DELETE /identity/v3/projects/613ceaf3719d459e956654325519a934 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:13:38.084094 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31cd97f-4ea4-4869-aeb1-35c51531557a service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:38.093632 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31cd97f-4ea4-4869-aeb1-35c51531557a service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:38.150002 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfebcb4-9a80-4ffa-8653-551c2ed948c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:38.150874 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfebcb4-9a80-4ffa-8653-551c2ed948c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:38.151159 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfebcb4-9a80-4ffa-8653-551c2ed948c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:38.151444 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfebcb4-9a80-4ffa-8653-551c2ed948c3 admin admin] PATH_INFO: `/v3/users/8c8c9adead804159ae6b3c6613412f70` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:38.152681 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfebcb4-9a80-4ffa-8653-551c2ed948c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c8c9adead804159ae6b3c6613412f70)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:38.154581 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfebcb4-9a80-4ffa-8653-551c2ed948c3 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:38.157095 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:13:38.333066 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8bfebcb4-9a80-4ffa-8653-551c2ed948c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c8c9adead804159ae6b3c6613412f70'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:38.342063 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8bfebcb4-9a80-4ffa-8653-551c2ed948c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c8c9adead804159ae6b3c6613412f70'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:38.432863 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8bfebcb4-9a80-4ffa-8653-551c2ed948c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c8c9adead804159ae6b3c6613412f70'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:38.433445 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8bfebcb4-9a80-4ffa-8653-551c2ed948c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c8c9adead804159ae6b3c6613412f70'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:38.446539 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8bfebcb4-9a80-4ffa-8653-551c2ed948c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c8c9adead804159ae6b3c6613412f70'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:38.452502 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1390/2760] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:13:38 2022] DELETE /identity/v3/users/8c8c9adead804159ae6b3c6613412f70 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:13:38.845718 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464c2a7c-309c-4455-8fbe-57a5594bbb0c admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:38.847147 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-464c2a7c-309c-4455-8fbe-57a5594bbb0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:13:38.868725 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464c2a7c-309c-4455-8fbe-57a5594bbb0c admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:38.933673 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485fced3-c009-4780-874b-d3496a392a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:38.934532 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485fced3-c009-4780-874b-d3496a392a70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:38.934818 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485fced3-c009-4780-874b-d3496a392a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:38.935125 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485fced3-c009-4780-874b-d3496a392a70 admin admin] PATH_INFO: `/v3/projects/01b21d696a4741fa9ba479157a1dfa5e` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:38.936353 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485fced3-c009-4780-874b-d3496a392a70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01b21d696a4741fa9ba479157a1dfa5e)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:38.938272 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485fced3-c009-4780-874b-d3496a392a70 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:39.020590 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-485fced3-c009-4780-874b-d3496a392a70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01b21d696a4741fa9ba479157a1dfa5e', 'request_id': 'req-485fced3-c009-4780-874b-d3496a392a70'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:39.025748 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-485fced3-c009-4780-874b-d3496a392a70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01b21d696a4741fa9ba479157a1dfa5e', 'request_id': 'req-485fced3-c009-4780-874b-d3496a392a70'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:39.031966 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-485fced3-c009-4780-874b-d3496a392a70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01b21d696a4741fa9ba479157a1dfa5e', 'request_id': 'req-485fced3-c009-4780-874b-d3496a392a70'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:39.088670 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-485fced3-c009-4780-874b-d3496a392a70 admin admin] The token cache is being invalidate because project 01b21d696a4741fa9ba479157a1dfa5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:13:39.088954 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-485fced3-c009-4780-874b-d3496a392a70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:39.089574 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-485fced3-c009-4780-874b-d3496a392a70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:39.091079 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1371/2761] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:13:38 2022] DELETE /identity/v3/projects/01b21d696a4741fa9ba479157a1dfa5e => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:13:39.096845 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6020dd1-0d9d-413d-8d52-90aedfd63426 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:39.097316 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6020dd1-0d9d-413d-8d52-90aedfd63426 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:39.097453 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6020dd1-0d9d-413d-8d52-90aedfd63426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:39.116010 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-c6020dd1-0d9d-413d-8d52-90aedfd63426 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:13:39.117204 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-c6020dd1-0d9d-413d-8d52-90aedfd63426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:13:39.176297 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1391/2762] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:13:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:13:39.184697 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5af704-ebc1-44ee-a510-b4e8372483cb None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:39.184986 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5af704-ebc1-44ee-a510-b4e8372483cb None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:39.185263 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5af704-ebc1-44ee-a510-b4e8372483cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:39.209732 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-5c5af704-ebc1-44ee-a510-b4e8372483cb None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:13:39.210951 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5c5af704-ebc1-44ee-a510-b4e8372483cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:13:39.218713 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1372/2763] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:13:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:13:39.232215 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6020dd1-0d9d-413d-8d52-90aedfd63426 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:39.242789 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6020dd1-0d9d-413d-8d52-90aedfd63426 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:39.248841 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428bbda3-784c-4a06-bcc0-1a97dea04261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:39.249504 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428bbda3-784c-4a06-bcc0-1a97dea04261 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:39.249785 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428bbda3-784c-4a06-bcc0-1a97dea04261 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:39.250061 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428bbda3-784c-4a06-bcc0-1a97dea04261 admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:39.250789 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428bbda3-784c-4a06-bcc0-1a97dea04261 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 13:13:39.251092 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428bbda3-784c-4a06-bcc0-1a97dea04261 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:39.252736 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428bbda3-784c-4a06-bcc0-1a97dea04261 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:39.265568 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1392/2764] 10.222.0.27 () {58 vars in 1152 bytes} [Fri Jun 10 13:13:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 13:13:39.269274 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5af704-ebc1-44ee-a510-b4e8372483cb None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:39.278779 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5af704-ebc1-44ee-a510-b4e8372483cb None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:39.284216 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf2df8e-1ac0-4fab-b5b6-833d535c0505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:39.284902 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf2df8e-1ac0-4fab-b5b6-833d535c0505 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:39.285327 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf2df8e-1ac0-4fab-b5b6-833d535c0505 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:39.285478 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf2df8e-1ac0-4fab-b5b6-833d535c0505 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:39.286175 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf2df8e-1ac0-4fab-b5b6-833d535c0505 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:39.288724 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf2df8e-1ac0-4fab-b5b6-833d535c0505 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:39.369543 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1373/2765] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:13:39 2022] POST /identity/v3/projects => generated 387 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:13:39.370936 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428bbda3-784c-4a06-bcc0-1a97dea04261 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:39.377130 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf2df8e-1ac0-4fab-b5b6-833d535c0505 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:39.378807 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-9bf2df8e-1ac0-4fab-b5b6-833d535c0505 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:13:39.382392 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428bbda3-784c-4a06-bcc0-1a97dea04261 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:39.401860 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:13:39.437328 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf2df8e-1ac0-4fab-b5b6-833d535c0505 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:39.439474 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ad649f-a7d1-4479-8e20-06faa957e560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:39.440239 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ad649f-a7d1-4479-8e20-06faa957e560 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:39.440545 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ad649f-a7d1-4479-8e20-06faa957e560 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:39.440824 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ad649f-a7d1-4479-8e20-06faa957e560 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:39.440972 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630d9c44-1668-43f2-8dba-963cab093170 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6cc95a563b941ac92ab283aa0d5a5d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:39.441544 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ad649f-a7d1-4479-8e20-06faa957e560 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:39.441657 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630d9c44-1668-43f2-8dba-963cab093170 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:39.441889 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630d9c44-1668-43f2-8dba-963cab093170 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:39.442168 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630d9c44-1668-43f2-8dba-963cab093170 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:39.444254 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ad649f-a7d1-4479-8e20-06faa957e560 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:39.444677 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-630d9c44-1668-43f2-8dba-963cab093170 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:13:39.516088 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630d9c44-1668-43f2-8dba-963cab093170 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:39.517522 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630d9c44-1668-43f2-8dba-963cab093170 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:39.528554 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1374/2766] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:13:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:13:39.553920 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:13:39.562868 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1393/2767] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:13:39 2022] POST /identity/v3/users => generated 294 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:13:39.564205 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630d9c44-1668-43f2-8dba-963cab093170 service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:39.575375 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630d9c44-1668-43f2-8dba-963cab093170 service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:39.580089 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0426d0ea-971b-48c3-8c11-4d2572f5f28d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:39.581000 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0426d0ea-971b-48c3-8c11-4d2572f5f28d admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:39.581286 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0426d0ea-971b-48c3-8c11-4d2572f5f28d admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:39.581560 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0426d0ea-971b-48c3-8c11-4d2572f5f28d admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:39.582280 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0426d0ea-971b-48c3-8c11-4d2572f5f28d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:39.584392 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0426d0ea-971b-48c3-8c11-4d2572f5f28d admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:39.594343 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1375/2768] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:13:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:13:39.598509 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ad649f-a7d1-4479-8e20-06faa957e560 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:39.609102 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ad649f-a7d1-4479-8e20-06faa957e560 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:39.613937 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef105b0-c55b-4e4a-9c3d-891d6a1a4ee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:39.614540 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef105b0-c55b-4e4a-9c3d-891d6a1a4ee5 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:39.615380 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef105b0-c55b-4e4a-9c3d-891d6a1a4ee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:39.615654 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef105b0-c55b-4e4a-9c3d-891d6a1a4ee5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:39.616429 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef105b0-c55b-4e4a-9c3d-891d6a1a4ee5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:39.626700 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef105b0-c55b-4e4a-9c3d-891d6a1a4ee5 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:39.628920 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1394/2769] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:13:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 10 13:13:39.633586 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0426d0ea-971b-48c3-8c11-4d2572f5f28d admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:39.643692 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0426d0ea-971b-48c3-8c11-4d2572f5f28d admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:39.649202 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a674f4-9ca3-4cd3-a294-92f8483b6c95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:39.650077 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a674f4-9ca3-4cd3-a294-92f8483b6c95 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:39.650371 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a674f4-9ca3-4cd3-a294-92f8483b6c95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:39.650666 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a674f4-9ca3-4cd3-a294-92f8483b6c95 admin admin] PATH_INFO: `/v3/projects/2e68124a0c5c4009b2b1ab9dc6ce3e0c/users/3f777d5ef365402b95198d6fe1a0b914/roles/43106c664e1b40de984c0c62701149c8` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:39.666165 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a674f4-9ca3-4cd3-a294-92f8483b6c95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e68124a0c5c4009b2b1ab9dc6ce3e0c, user_id=3f777d5ef365402b95198d6fe1a0b914, role_id=43106c664e1b40de984c0c62701149c8)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:39.668822 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a674f4-9ca3-4cd3-a294-92f8483b6c95 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:39.747380 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1376/2770] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:13:39 2022] PUT /identity/v3/projects/2e68124a0c5c4009b2b1ab9dc6ce3e0c/users/3f777d5ef365402b95198d6fe1a0b914/roles/43106c664e1b40de984c0c62701149c8 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:13:39.758273 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158f544b-ae3b-4994-9e65-c4d325d71c3a None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:39.758567 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158f544b-ae3b-4994-9e65-c4d325d71c3a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:39.758847 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158f544b-ae3b-4994-9e65-c4d325d71c3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:39.788538 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-158f544b-ae3b-4994-9e65-c4d325d71c3a None None] MFA Rules not processed for user `3f777d5ef365402b95198d6fe1a0b914`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:13:39.789735 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-158f544b-ae3b-4994-9e65-c4d325d71c3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:13:39.866470 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1395/2771] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:13:39 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:13:39.870840 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a674f4-9ca3-4cd3-a294-92f8483b6c95 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:39.936875 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a674f4-9ca3-4cd3-a294-92f8483b6c95 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:39.941462 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5577ff0-6fed-425c-a901-55a9e671aadb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:39.942115 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5577ff0-6fed-425c-a901-55a9e671aadb admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:39.942400 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5577ff0-6fed-425c-a901-55a9e671aadb admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:39.942770 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5577ff0-6fed-425c-a901-55a9e671aadb admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:39.943400 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5577ff0-6fed-425c-a901-55a9e671aadb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:39.945365 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5577ff0-6fed-425c-a901-55a9e671aadb admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:40.011816 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1377/2772] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:13:39 2022] POST /identity/v3/projects => generated 385 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:13:40.015888 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158f544b-ae3b-4994-9e65-c4d325d71c3a None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:40.027183 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158f544b-ae3b-4994-9e65-c4d325d71c3a None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:40.088367 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae09ce9-12cd-47c0-ba74-4cdd1c17cef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:40.089004 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae09ce9-12cd-47c0-ba74-4cdd1c17cef7 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:40.089294 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae09ce9-12cd-47c0-ba74-4cdd1c17cef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:40.089576 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae09ce9-12cd-47c0-ba74-4cdd1c17cef7 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:40.090266 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae09ce9-12cd-47c0-ba74-4cdd1c17cef7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:40.092225 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae09ce9-12cd-47c0-ba74-4cdd1c17cef7 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:40.147484 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1396/2773] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:13:40 2022] POST /identity/v3/users => generated 299 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:13:40.151521 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5577ff0-6fed-425c-a901-55a9e671aadb admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:40.162916 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5577ff0-6fed-425c-a901-55a9e671aadb admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:40.167388 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64a8bc7-d52e-41a8-bdba-56a8e32625a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:40.168287 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64a8bc7-d52e-41a8-bdba-56a8e32625a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:40.168571 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64a8bc7-d52e-41a8-bdba-56a8e32625a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:40.168850 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64a8bc7-d52e-41a8-bdba-56a8e32625a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:40.169578 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64a8bc7-d52e-41a8-bdba-56a8e32625a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:40.171270 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64a8bc7-d52e-41a8-bdba-56a8e32625a0 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:40.176795 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:13:40.182330 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1378/2774] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:13:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:13:40.186687 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae09ce9-12cd-47c0-ba74-4cdd1c17cef7 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:40.196721 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae09ce9-12cd-47c0-ba74-4cdd1c17cef7 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:40.201335 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8911be9-997d-4156-a6d4-828f1fa6d35f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:40.202247 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8911be9-997d-4156-a6d4-828f1fa6d35f admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:40.202534 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8911be9-997d-4156-a6d4-828f1fa6d35f admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:40.202818 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8911be9-997d-4156-a6d4-828f1fa6d35f admin admin] PATH_INFO: `/v3/projects/0248aa2032944c3ab5169b89197ccac0/users/796d65d621364d8091d25a22a926d4f3/roles/b43d5a8247e244869c2c70f986f42e8a` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:40.219207 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8911be9-997d-4156-a6d4-828f1fa6d35f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0248aa2032944c3ab5169b89197ccac0, user_id=796d65d621364d8091d25a22a926d4f3, role_id=b43d5a8247e244869c2c70f986f42e8a)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:40.221889 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8911be9-997d-4156-a6d4-828f1fa6d35f admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:40.324888 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1397/2775] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:13:40 2022] PUT /identity/v3/projects/0248aa2032944c3ab5169b89197ccac0/users/796d65d621364d8091d25a22a926d4f3/roles/b43d5a8247e244869c2c70f986f42e8a => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:13:40.331230 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64a8bc7-d52e-41a8-bdba-56a8e32625a0 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:40.341851 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64a8bc7-d52e-41a8-bdba-56a8e32625a0 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:40.400391 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d1810d-7e01-403a-b575-82ccd56b7ce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:40.401056 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d1810d-7e01-403a-b575-82ccd56b7ce5 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:40.401338 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d1810d-7e01-403a-b575-82ccd56b7ce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:40.401612 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d1810d-7e01-403a-b575-82ccd56b7ce5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:40.402334 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d1810d-7e01-403a-b575-82ccd56b7ce5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:40.403958 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d1810d-7e01-403a-b575-82ccd56b7ce5 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:40.414784 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1379/2776] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:13:40 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:13:40.419190 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8911be9-997d-4156-a6d4-828f1fa6d35f admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:40.430383 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8911be9-997d-4156-a6d4-828f1fa6d35f admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:40.434984 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9177d08-8565-48e6-b60b-6fde7750e113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:40.435873 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9177d08-8565-48e6-b60b-6fde7750e113 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:40.436167 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9177d08-8565-48e6-b60b-6fde7750e113 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:40.436675 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9177d08-8565-48e6-b60b-6fde7750e113 admin admin] PATH_INFO: `/v3/projects/0248aa2032944c3ab5169b89197ccac0/users/796d65d621364d8091d25a22a926d4f3/roles/b43d5a8247e244869c2c70f986f42e8a` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:40.438645 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9177d08-8565-48e6-b60b-6fde7750e113 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0248aa2032944c3ab5169b89197ccac0, user_id=796d65d621364d8091d25a22a926d4f3, role_id=b43d5a8247e244869c2c70f986f42e8a)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:40.441239 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9177d08-8565-48e6-b60b-6fde7750e113 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:40.451806 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1398/2777] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:13:40 2022] PUT /identity/v3/projects/0248aa2032944c3ab5169b89197ccac0/users/796d65d621364d8091d25a22a926d4f3/roles/b43d5a8247e244869c2c70f986f42e8a => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 10 13:13:40.463210 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eba14a-dcc6-46de-b6d9-4e50af60d884 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:40.463506 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eba14a-dcc6-46de-b6d9-4e50af60d884 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:40.463792 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7eba14a-dcc6-46de-b6d9-4e50af60d884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:40.491655 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-b7eba14a-dcc6-46de-b6d9-4e50af60d884 None None] MFA Rules not processed for user `796d65d621364d8091d25a22a926d4f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:13:40.492901 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b7eba14a-dcc6-46de-b6d9-4e50af60d884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:13:40.560524 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1380/2778] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:13:40 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:13:40.572104 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9177d08-8565-48e6-b60b-6fde7750e113 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:40.603587 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:13:40.627038 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9177d08-8565-48e6-b60b-6fde7750e113 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:40.631428 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f223fb-d8aa-40ba-b4f2-534add439c58 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6cc95a563b941ac92ab283aa0d5a5d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:40.632094 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f223fb-d8aa-40ba-b4f2-534add439c58 service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:40.632376 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f223fb-d8aa-40ba-b4f2-534add439c58 service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:40.632653 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f223fb-d8aa-40ba-b4f2-534add439c58 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:40.641816 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f223fb-d8aa-40ba-b4f2-534add439c58 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:40.643421 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f223fb-d8aa-40ba-b4f2-534add439c58 service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:40.653793 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1399/2779] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:13:40 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:13:40.717521 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eba14a-dcc6-46de-b6d9-4e50af60d884 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:40.780639 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7eba14a-dcc6-46de-b6d9-4e50af60d884 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:40.785097 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea290ff1-9a72-4a59-bf0b-0a411e155b40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:40.785960 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea290ff1-9a72-4a59-bf0b-0a411e155b40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:40.786267 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea290ff1-9a72-4a59-bf0b-0a411e155b40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:40.787045 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea290ff1-9a72-4a59-bf0b-0a411e155b40 admin admin] PATH_INFO: `/v3/users/3f777d5ef365402b95198d6fe1a0b914` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:40.788279 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea290ff1-9a72-4a59-bf0b-0a411e155b40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f777d5ef365402b95198d6fe1a0b914)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:40.790168 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea290ff1-9a72-4a59-bf0b-0a411e155b40 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:40.812639 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:13:40.960320 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ea290ff1-9a72-4a59-bf0b-0a411e155b40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f777d5ef365402b95198d6fe1a0b914'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:40.967223 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ea290ff1-9a72-4a59-bf0b-0a411e155b40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f777d5ef365402b95198d6fe1a0b914'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:41.066837 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ea290ff1-9a72-4a59-bf0b-0a411e155b40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f777d5ef365402b95198d6fe1a0b914'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:41.067452 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ea290ff1-9a72-4a59-bf0b-0a411e155b40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f777d5ef365402b95198d6fe1a0b914'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:41.079802 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-ea290ff1-9a72-4a59-bf0b-0a411e155b40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f777d5ef365402b95198d6fe1a0b914'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:41.088482 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1381/2780] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:13:40 2022] DELETE /identity/v3/users/3f777d5ef365402b95198d6fe1a0b914 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:13:41.091204 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f63cbc-61cb-4702-b9aa-503995cbc68a None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:41.091491 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f63cbc-61cb-4702-b9aa-503995cbc68a None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:41.091764 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f63cbc-61cb-4702-b9aa-503995cbc68a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:41.109977 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-17f63cbc-61cb-4702-b9aa-503995cbc68a None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:13:41.111197 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-17f63cbc-61cb-4702-b9aa-503995cbc68a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:13:41.170357 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1400/2781] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:13:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:13:41.180048 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea290ff1-9a72-4a59-bf0b-0a411e155b40 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:41.181457 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-ea290ff1-9a72-4a59-bf0b-0a411e155b40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:13:41.238115 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea290ff1-9a72-4a59-bf0b-0a411e155b40 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:41.241685 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c34aa1-608a-4b5b-97c2-9675645b43ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:41.242314 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c34aa1-608a-4b5b-97c2-9675645b43ee service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:41.242596 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c34aa1-608a-4b5b-97c2-9675645b43ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:41.242870 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c34aa1-608a-4b5b-97c2-9675645b43ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:41.251672 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c34aa1-608a-4b5b-97c2-9675645b43ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:41.253128 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c34aa1-608a-4b5b-97c2-9675645b43ee service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:41.262789 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1382/2782] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:13:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:13:41.686281 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f63cbc-61cb-4702-b9aa-503995cbc68a None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:41.687872 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-17f63cbc-61cb-4702-b9aa-503995cbc68a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:13:41.701894 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f63cbc-61cb-4702-b9aa-503995cbc68a None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:41.706972 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c84844-4824-454d-907a-9b635097190c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:41.707882 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c84844-4824-454d-907a-9b635097190c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:41.708218 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c84844-4824-454d-907a-9b635097190c admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:41.708583 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c84844-4824-454d-907a-9b635097190c admin admin] PATH_INFO: `/v3/projects/2e68124a0c5c4009b2b1ab9dc6ce3e0c` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:41.709807 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c84844-4824-454d-907a-9b635097190c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e68124a0c5c4009b2b1ab9dc6ce3e0c)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:41.711945 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c84844-4824-454d-907a-9b635097190c admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:41.810345 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-07c84844-4824-454d-907a-9b635097190c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e68124a0c5c4009b2b1ab9dc6ce3e0c', 'request_id': 'req-07c84844-4824-454d-907a-9b635097190c'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:41.815972 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-07c84844-4824-454d-907a-9b635097190c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e68124a0c5c4009b2b1ab9dc6ce3e0c', 'request_id': 'req-07c84844-4824-454d-907a-9b635097190c'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:41.822960 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-07c84844-4824-454d-907a-9b635097190c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e68124a0c5c4009b2b1ab9dc6ce3e0c', 'request_id': 'req-07c84844-4824-454d-907a-9b635097190c'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:41.877448 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-07c84844-4824-454d-907a-9b635097190c admin admin] The token cache is being invalidate because project 2e68124a0c5c4009b2b1ab9dc6ce3e0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:13:41.877741 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-07c84844-4824-454d-907a-9b635097190c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:41.878354 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-07c84844-4824-454d-907a-9b635097190c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:41.879912 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1401/2783] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:13:41 2022] DELETE /identity/v3/projects/2e68124a0c5c4009b2b1ab9dc6ce3e0c => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:13:41.895806 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c34aa1-608a-4b5b-97c2-9675645b43ee service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:41.906131 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c34aa1-608a-4b5b-97c2-9675645b43ee service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:41.961542 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345117a5-50e2-448f-bb05-9f9e8076e253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:41.962326 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345117a5-50e2-448f-bb05-9f9e8076e253 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:41.962608 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345117a5-50e2-448f-bb05-9f9e8076e253 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:41.962893 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345117a5-50e2-448f-bb05-9f9e8076e253 admin admin] PATH_INFO: `/v3/users/796d65d621364d8091d25a22a926d4f3` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:41.964072 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345117a5-50e2-448f-bb05-9f9e8076e253 admin admin] RBAC: Authorizing `identity:delete_user(user_id=796d65d621364d8091d25a22a926d4f3)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:41.965961 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345117a5-50e2-448f-bb05-9f9e8076e253 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:42.126129 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:13:42.127679 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-345117a5-50e2-448f-bb05-9f9e8076e253 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '796d65d621364d8091d25a22a926d4f3'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:42.135293 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-345117a5-50e2-448f-bb05-9f9e8076e253 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '796d65d621364d8091d25a22a926d4f3'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:42.204870 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-345117a5-50e2-448f-bb05-9f9e8076e253 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '796d65d621364d8091d25a22a926d4f3'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:42.205763 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-345117a5-50e2-448f-bb05-9f9e8076e253 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '796d65d621364d8091d25a22a926d4f3'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:42.222213 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-345117a5-50e2-448f-bb05-9f9e8076e253 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '796d65d621364d8091d25a22a926d4f3'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:42.228365 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1383/2784] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:13:41 2022] DELETE /identity/v3/users/796d65d621364d8091d25a22a926d4f3 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:13:42.630297 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c84844-4824-454d-907a-9b635097190c admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:42.630297 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-07c84844-4824-454d-907a-9b635097190c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:13:42.643275 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c84844-4824-454d-907a-9b635097190c admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:42.694396 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:13:42.700885 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee1d845-384b-4b9e-ab8a-81b442e65457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:42.701743 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee1d845-384b-4b9e-ab8a-81b442e65457 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:42.702030 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee1d845-384b-4b9e-ab8a-81b442e65457 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:42.702311 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee1d845-384b-4b9e-ab8a-81b442e65457 admin admin] PATH_INFO: `/v3/projects/0248aa2032944c3ab5169b89197ccac0` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:42.703497 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee1d845-384b-4b9e-ab8a-81b442e65457 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0248aa2032944c3ab5169b89197ccac0)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:42.705528 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee1d845-384b-4b9e-ab8a-81b442e65457 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:42.817106 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-1ee1d845-384b-4b9e-ab8a-81b442e65457 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0248aa2032944c3ab5169b89197ccac0', 'request_id': 'req-1ee1d845-384b-4b9e-ab8a-81b442e65457'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:42.822461 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-1ee1d845-384b-4b9e-ab8a-81b442e65457 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0248aa2032944c3ab5169b89197ccac0', 'request_id': 'req-1ee1d845-384b-4b9e-ab8a-81b442e65457'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:42.828722 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-1ee1d845-384b-4b9e-ab8a-81b442e65457 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0248aa2032944c3ab5169b89197ccac0', 'request_id': 'req-1ee1d845-384b-4b9e-ab8a-81b442e65457'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:42.883543 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-1ee1d845-384b-4b9e-ab8a-81b442e65457 admin admin] The token cache is being invalidate because project 0248aa2032944c3ab5169b89197ccac0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:13:42.886174 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-1ee1d845-384b-4b9e-ab8a-81b442e65457 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:42.890245 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-1ee1d845-384b-4b9e-ab8a-81b442e65457 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:42.895042 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1402/2785] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:13:42 2022] DELETE /identity/v3/projects/0248aa2032944c3ab5169b89197ccac0 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:13:42.896916 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dd0df5-19d7-4b05-935b-9847965df03f None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:42.897863 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dd0df5-19d7-4b05-935b-9847965df03f None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:42.897863 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dd0df5-19d7-4b05-935b-9847965df03f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:42.921777 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-15dd0df5-19d7-4b05-935b-9847965df03f None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:13:42.922974 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-15dd0df5-19d7-4b05-935b-9847965df03f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:13:42.983075 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1384/2786] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:13:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:13:42.991780 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e3a57-48d0-405f-ad4c-0f8c96371e00 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:42.992185 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e3a57-48d0-405f-ad4c-0f8c96371e00 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:42.992483 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2e3a57-48d0-405f-ad4c-0f8c96371e00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:43.010544 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-0e2e3a57-48d0-405f-ad4c-0f8c96371e00 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:13:43.011754 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0e2e3a57-48d0-405f-ad4c-0f8c96371e00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:13:43.019522 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1403/2787] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:13:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:13:43.023963 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dd0df5-19d7-4b05-935b-9847965df03f None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:43.036526 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dd0df5-19d7-4b05-935b-9847965df03f None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:43.039586 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd23b4b-229d-4ec1-829b-e44ff1dd8972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:43.040304 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd23b4b-229d-4ec1-829b-e44ff1dd8972 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:43.040594 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd23b4b-229d-4ec1-829b-e44ff1dd8972 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:43.040871 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd23b4b-229d-4ec1-829b-e44ff1dd8972 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:43.041600 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd23b4b-229d-4ec1-829b-e44ff1dd8972 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 13:13:43.041911 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd23b4b-229d-4ec1-829b-e44ff1dd8972 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:43.043555 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd23b4b-229d-4ec1-829b-e44ff1dd8972 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:43.056253 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1385/2788] 10.222.0.27 () {58 vars in 1152 bytes} [Fri Jun 10 13:13:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 13:13:43.060532 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2e3a57-48d0-405f-ad4c-0f8c96371e00 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:43.069720 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2e3a57-48d0-405f-ad4c-0f8c96371e00 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:43.074276 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a428327d-f4ca-47d9-a0e6-96583264ecbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:43.074918 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a428327d-f4ca-47d9-a0e6-96583264ecbc admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:43.075199 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a428327d-f4ca-47d9-a0e6-96583264ecbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:43.075476 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a428327d-f4ca-47d9-a0e6-96583264ecbc admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:43.076207 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a428327d-f4ca-47d9-a0e6-96583264ecbc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:43.078139 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a428327d-f4ca-47d9-a0e6-96583264ecbc admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:43.182118 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1404/2789] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:13:43 2022] POST /identity/v3/projects => generated 373 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:13:43.186617 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd23b4b-229d-4ec1-829b-e44ff1dd8972 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:43.196811 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd23b4b-229d-4ec1-829b-e44ff1dd8972 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:43.251982 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a13c21-0097-4f96-8e66-041d17d7881e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:43.252644 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a13c21-0097-4f96-8e66-041d17d7881e admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:43.252930 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a13c21-0097-4f96-8e66-041d17d7881e admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:43.253210 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a13c21-0097-4f96-8e66-041d17d7881e admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:43.253940 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a13c21-0097-4f96-8e66-041d17d7881e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:43.255855 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a13c21-0097-4f96-8e66-041d17d7881e admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:43.353378 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1386/2790] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:13:43 2022] POST /identity/v3/users => generated 287 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:13:43.357253 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a428327d-f4ca-47d9-a0e6-96583264ecbc admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:43.367872 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a428327d-f4ca-47d9-a0e6-96583264ecbc admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:43.372419 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba63c190-d700-4cfb-8936-0f27abbaeb1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:43.373094 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba63c190-d700-4cfb-8936-0f27abbaeb1a admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:43.373378 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba63c190-d700-4cfb-8936-0f27abbaeb1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:43.373654 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba63c190-d700-4cfb-8936-0f27abbaeb1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:43.374365 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba63c190-d700-4cfb-8936-0f27abbaeb1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:43.375967 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba63c190-d700-4cfb-8936-0f27abbaeb1a admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:43.386666 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1405/2791] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:13:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:13:43.390597 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a13c21-0097-4f96-8e66-041d17d7881e admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:43.400581 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a13c21-0097-4f96-8e66-041d17d7881e admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:43.405448 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ab9161-32f2-48ca-be05-302d083b8722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:43.406098 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ab9161-32f2-48ca-be05-302d083b8722 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:43.406381 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ab9161-32f2-48ca-be05-302d083b8722 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:43.406655 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ab9161-32f2-48ca-be05-302d083b8722 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:43.407378 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ab9161-32f2-48ca-be05-302d083b8722 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:43.409019 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ab9161-32f2-48ca-be05-302d083b8722 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:43.419668 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1387/2792] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:13:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:13:43.423561 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba63c190-d700-4cfb-8936-0f27abbaeb1a admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:43.433777 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba63c190-d700-4cfb-8936-0f27abbaeb1a admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:43.438255 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c29a085-0bba-4ce8-b116-14eba8c0d661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:43.439122 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c29a085-0bba-4ce8-b116-14eba8c0d661 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:43.439402 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c29a085-0bba-4ce8-b116-14eba8c0d661 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:43.439679 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c29a085-0bba-4ce8-b116-14eba8c0d661 admin admin] PATH_INFO: `/v3/projects/94ba1ecd5b9e49108a46d7064122de04/users/8b04b18237334532afda8607312c973f/roles/43106c664e1b40de984c0c62701149c8` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:43.454676 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c29a085-0bba-4ce8-b116-14eba8c0d661 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94ba1ecd5b9e49108a46d7064122de04, user_id=8b04b18237334532afda8607312c973f, role_id=43106c664e1b40de984c0c62701149c8)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:43.457304 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c29a085-0bba-4ce8-b116-14eba8c0d661 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:43.506788 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1406/2793] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:13:43 2022] PUT /identity/v3/projects/94ba1ecd5b9e49108a46d7064122de04/users/8b04b18237334532afda8607312c973f/roles/43106c664e1b40de984c0c62701149c8 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:13:43.511738 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f7b957-d1af-405d-b81d-427d054187b1 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:43.512063 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f7b957-d1af-405d-b81d-427d054187b1 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:43.512405 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f7b957-d1af-405d-b81d-427d054187b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:43.518486 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef592e3-cc00-4a7d-b0dc-0a8ddffb1bf3 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:43.520198 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef592e3-cc00-4a7d-b0dc-0a8ddffb1bf3 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:43.520198 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef592e3-cc00-4a7d-b0dc-0a8ddffb1bf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:43.529499 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-03f7b957-d1af-405d-b81d-427d054187b1 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:13:43.531009 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-03f7b957-d1af-405d-b81d-427d054187b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:13:43.536646 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-fef592e3-cc00-4a7d-b0dc-0a8ddffb1bf3 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:13:43.538405 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-fef592e3-cc00-4a7d-b0dc-0a8ddffb1bf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:13:43.557632 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:13:43.591218 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1388/2794] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:13:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:13:43.600633 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1407/2795] 10.222.0.36 () {58 vars in 1016 bytes} [Fri Jun 10 13:13:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 10 13:13:43.601747 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f7b957-d1af-405d-b81d-427d054187b1 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:43.603130 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-03f7b957-d1af-405d-b81d-427d054187b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:13:43.622781 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef592e3-cc00-4a7d-b0dc-0a8ddffb1bf3 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:43.624152 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-fef592e3-cc00-4a7d-b0dc-0a8ddffb1bf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:13:43.649381 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:13:43.661744 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f7b957-d1af-405d-b81d-427d054187b1 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:43.666006 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd808386-c3d2-4fe7-9ca0-fb8bb90881c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:43.666658 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd808386-c3d2-4fe7-9ca0-fb8bb90881c5 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:43.666946 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd808386-c3d2-4fe7-9ca0-fb8bb90881c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:43.667220 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd808386-c3d2-4fe7-9ca0-fb8bb90881c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:43.677887 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd808386-c3d2-4fe7-9ca0-fb8bb90881c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:43.680146 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd808386-c3d2-4fe7-9ca0-fb8bb90881c5 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:43.689792 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1389/2796] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:13:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:13:43.696964 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef592e3-cc00-4a7d-b0dc-0a8ddffb1bf3 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:43.703958 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17cd75d-307e-4507-9eff-939d446e31da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bd6caa461e84100a3d5fe5af5d80f3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:43.704617 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17cd75d-307e-4507-9eff-939d446e31da service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:43.704903 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17cd75d-307e-4507-9eff-939d446e31da service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:43.705183 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17cd75d-307e-4507-9eff-939d446e31da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:43.713809 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17cd75d-307e-4507-9eff-939d446e31da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:43.715380 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17cd75d-307e-4507-9eff-939d446e31da service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:43.724703 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1408/2797] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:13:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 10 13:13:44.794573 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5798cc04-e974-4c3d-b5a1-f39839c60bd3 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:44.795122 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5798cc04-e974-4c3d-b5a1-f39839c60bd3 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:44.795266 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5798cc04-e974-4c3d-b5a1-f39839c60bd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:44.821939 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-5798cc04-e974-4c3d-b5a1-f39839c60bd3 None None] MFA Rules not processed for user `8b04b18237334532afda8607312c973f`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:13:44.823147 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-5798cc04-e974-4c3d-b5a1-f39839c60bd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:13:44.895178 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1390/2798] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:13:44 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:13:44.900220 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17cd75d-307e-4507-9eff-939d446e31da service nova] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:44.910273 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17cd75d-307e-4507-9eff-939d446e31da service nova] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:44.915828 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fd4caa-6c2b-4e04-ba24-4250ec2f7ad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:44.916531 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fd4caa-6c2b-4e04-ba24-4250ec2f7ad0 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:44.916817 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fd4caa-6c2b-4e04-ba24-4250ec2f7ad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:44.917152 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fd4caa-6c2b-4e04-ba24-4250ec2f7ad0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:44.917810 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fd4caa-6c2b-4e04-ba24-4250ec2f7ad0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:44.919896 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fd4caa-6c2b-4e04-ba24-4250ec2f7ad0 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:45.012228 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1409/2799] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:13:44 2022] POST /identity/v3/projects => generated 373 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:13:45.016353 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5798cc04-e974-4c3d-b5a1-f39839c60bd3 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:45.027747 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5798cc04-e974-4c3d-b5a1-f39839c60bd3 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:45.081916 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54515d6-88f1-4ac4-82cd-2baa39713cbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:45.082514 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54515d6-88f1-4ac4-82cd-2baa39713cbd admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:45.082800 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54515d6-88f1-4ac4-82cd-2baa39713cbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:45.083080 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54515d6-88f1-4ac4-82cd-2baa39713cbd admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:45.083799 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54515d6-88f1-4ac4-82cd-2baa39713cbd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:45.086561 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54515d6-88f1-4ac4-82cd-2baa39713cbd admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:45.094260 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:13:45.171002 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1391/2800] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:13:45 2022] POST /identity/v3/users => generated 293 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 10 13:13:45.174918 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fd4caa-6c2b-4e04-ba24-4250ec2f7ad0 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:45.185341 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fd4caa-6c2b-4e04-ba24-4250ec2f7ad0 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:45.190030 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40730c7d-a907-4c95-bf7f-289185cfb673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:45.190686 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40730c7d-a907-4c95-bf7f-289185cfb673 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:45.190976 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40730c7d-a907-4c95-bf7f-289185cfb673 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:45.191255 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40730c7d-a907-4c95-bf7f-289185cfb673 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:45.192063 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40730c7d-a907-4c95-bf7f-289185cfb673 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:45.194149 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40730c7d-a907-4c95-bf7f-289185cfb673 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:45.205801 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1410/2801] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:13:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:13:45.210599 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54515d6-88f1-4ac4-82cd-2baa39713cbd admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:45.220758 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54515d6-88f1-4ac4-82cd-2baa39713cbd admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:45.225391 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15041480-c300-476f-a2f0-028a2dad14b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:45.226280 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15041480-c300-476f-a2f0-028a2dad14b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:45.226786 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15041480-c300-476f-a2f0-028a2dad14b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:45.227301 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15041480-c300-476f-a2f0-028a2dad14b0 admin admin] PATH_INFO: `/v3/projects/e3d144b60d1042958ff751b86bf6c53b/users/e891db98760447cda84a31c715a52234/roles/b43d5a8247e244869c2c70f986f42e8a` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:45.242486 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15041480-c300-476f-a2f0-028a2dad14b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3d144b60d1042958ff751b86bf6c53b, user_id=e891db98760447cda84a31c715a52234, role_id=b43d5a8247e244869c2c70f986f42e8a)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:45.245143 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15041480-c300-476f-a2f0-028a2dad14b0 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:45.318186 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1392/2802] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:13:45 2022] PUT /identity/v3/projects/e3d144b60d1042958ff751b86bf6c53b/users/e891db98760447cda84a31c715a52234/roles/b43d5a8247e244869c2c70f986f42e8a => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:13:45.322600 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40730c7d-a907-4c95-bf7f-289185cfb673 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:45.332564 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40730c7d-a907-4c95-bf7f-289185cfb673 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:45.388452 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691e28ec-4ac8-4e83-b357-f1f0bb65138f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:45.389104 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691e28ec-4ac8-4e83-b357-f1f0bb65138f admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:45.389389 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691e28ec-4ac8-4e83-b357-f1f0bb65138f admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:45.389697 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691e28ec-4ac8-4e83-b357-f1f0bb65138f admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:45.390413 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691e28ec-4ac8-4e83-b357-f1f0bb65138f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:45.392053 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691e28ec-4ac8-4e83-b357-f1f0bb65138f admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:45.404705 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1411/2803] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:13:45 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:13:45.406038 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15041480-c300-476f-a2f0-028a2dad14b0 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:45.416374 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15041480-c300-476f-a2f0-028a2dad14b0 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:45.421298 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c4abd1-e9e5-4af2-8604-5d24b912714f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:45.422205 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c4abd1-e9e5-4af2-8604-5d24b912714f admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:45.422488 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c4abd1-e9e5-4af2-8604-5d24b912714f admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:45.422765 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c4abd1-e9e5-4af2-8604-5d24b912714f admin admin] PATH_INFO: `/v3/projects/e3d144b60d1042958ff751b86bf6c53b/users/e891db98760447cda84a31c715a52234/roles/b43d5a8247e244869c2c70f986f42e8a` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:45.424790 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c4abd1-e9e5-4af2-8604-5d24b912714f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3d144b60d1042958ff751b86bf6c53b, user_id=e891db98760447cda84a31c715a52234, role_id=b43d5a8247e244869c2c70f986f42e8a)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:45.427422 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c4abd1-e9e5-4af2-8604-5d24b912714f admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:45.438165 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1393/2804] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:13:45 2022] PUT /identity/v3/projects/e3d144b60d1042958ff751b86bf6c53b/users/e891db98760447cda84a31c715a52234/roles/b43d5a8247e244869c2c70f986f42e8a => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 10 13:13:46.505351 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5005166-f196-4e0e-b83e-ead61f0c5ef0 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:46.506111 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5005166-f196-4e0e-b83e-ead61f0c5ef0 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:46.506111 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5005166-f196-4e0e-b83e-ead61f0c5ef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:46.533342 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-b5005166-f196-4e0e-b83e-ead61f0c5ef0 None None] MFA Rules not processed for user `e891db98760447cda84a31c715a52234`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:13:46.534582 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b5005166-f196-4e0e-b83e-ead61f0c5ef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:13:46.601326 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:13:46.611543 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1412/2805] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:13:46 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:13:46.623569 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bc766a-7d2a-493b-8d00-717c314d6fe2 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:46.624211 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bc766a-7d2a-493b-8d00-717c314d6fe2 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:46.624211 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bc766a-7d2a-493b-8d00-717c314d6fe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:46.643185 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-b9bc766a-7d2a-493b-8d00-717c314d6fe2 None None] MFA Rules not processed for user `8b04b18237334532afda8607312c973f`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:13:46.643185 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b9bc766a-7d2a-493b-8d00-717c314d6fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:13:46.697576 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1394/2806] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:13:46 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:13:46.708714 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5005166-f196-4e0e-b83e-ead61f0c5ef0 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:46.710169 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b5005166-f196-4e0e-b83e-ead61f0c5ef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:13:46.781926 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5005166-f196-4e0e-b83e-ead61f0c5ef0 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:46.787932 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f85b75a-6766-4eb8-8e08-6956b3e53ca3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bd6caa461e84100a3d5fe5af5d80f3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:46.788570 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f85b75a-6766-4eb8-8e08-6956b3e53ca3 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:46.788853 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f85b75a-6766-4eb8-8e08-6956b3e53ca3 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:46.789129 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f85b75a-6766-4eb8-8e08-6956b3e53ca3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:46.796785 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f85b75a-6766-4eb8-8e08-6956b3e53ca3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:46.798346 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f85b75a-6766-4eb8-8e08-6956b3e53ca3 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:46.807609 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1413/2807] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:13:46 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:13:47.423602 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bc766a-7d2a-493b-8d00-717c314d6fe2 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:47.425223 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b9bc766a-7d2a-493b-8d00-717c314d6fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:13:47.430977 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f85b75a-6766-4eb8-8e08-6956b3e53ca3 service nova] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:47.432605 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0f85b75a-6766-4eb8-8e08-6956b3e53ca3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:13:47.496724 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f85b75a-6766-4eb8-8e08-6956b3e53ca3 service nova] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:47.498058 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bc766a-7d2a-493b-8d00-717c314d6fe2 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:47.500325 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdd5d4e-4d9a-4bb8-b750-3c19f89f0efe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:47.501035 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdd5d4e-4d9a-4bb8-b750-3c19f89f0efe service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:47.501389 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdd5d4e-4d9a-4bb8-b750-3c19f89f0efe service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:47.501738 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdd5d4e-4d9a-4bb8-b750-3c19f89f0efe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:47.503075 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f52544-e6c3-4c81-9ece-fadb15870d5c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd972ab782b3f4d2aba575314f16dafc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:47.503617 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-7cdd5d4e-4d9a-4bb8-b750-3c19f89f0efe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:13:47.503723 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f52544-e6c3-4c81-9ece-fadb15870d5c service placement] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:47.504022 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f52544-e6c3-4c81-9ece-fadb15870d5c service placement] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:47.504304 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f52544-e6c3-4c81-9ece-fadb15870d5c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:47.506030 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c7f52544-e6c3-4c81-9ece-fadb15870d5c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:13:47.517297 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdd5d4e-4d9a-4bb8-b750-3c19f89f0efe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:47.517806 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f52544-e6c3-4c81-9ece-fadb15870d5c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:47.518726 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdd5d4e-4d9a-4bb8-b750-3c19f89f0efe service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:47.519109 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f52544-e6c3-4c81-9ece-fadb15870d5c service placement] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:47.529447 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1395/2808] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:13:47 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:13:47.529967 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1414/2809] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:13:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 10 13:13:48.723912 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae960dc-d9c4-407d-b504-cd977b170712 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:48.724362 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae960dc-d9c4-407d-b504-cd977b170712 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:48.724463 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae960dc-d9c4-407d-b504-cd977b170712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:48.743311 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-0ae960dc-d9c4-407d-b504-cd977b170712 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:13:48.744533 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0ae960dc-d9c4-407d-b504-cd977b170712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:13:48.790310 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:13:48.805962 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1396/2810] 10.222.0.36 () {58 vars in 1016 bytes} [Fri Jun 10 13:13:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:13:48.825879 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdd5d4e-4d9a-4bb8-b750-3c19f89f0efe service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:48.837099 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdd5d4e-4d9a-4bb8-b750-3c19f89f0efe service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:48.842789 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511b58cc-1c0b-47c3-b034-0f1c9a9bea64 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bd6caa461e84100a3d5fe5af5d80f3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:48.843391 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511b58cc-1c0b-47c3-b034-0f1c9a9bea64 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:48.843682 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511b58cc-1c0b-47c3-b034-0f1c9a9bea64 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:48.844031 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511b58cc-1c0b-47c3-b034-0f1c9a9bea64 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:48.853359 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511b58cc-1c0b-47c3-b034-0f1c9a9bea64 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:48.855074 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511b58cc-1c0b-47c3-b034-0f1c9a9bea64 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:48.871604 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1415/2811] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:13:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:13:51.747607 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae960dc-d9c4-407d-b504-cd977b170712 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:51.757596 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae960dc-d9c4-407d-b504-cd977b170712 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:51.761094 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472c4872-56dd-450f-b295-e44224f1bd35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:51.761725 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472c4872-56dd-450f-b295-e44224f1bd35 service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:51.762004 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472c4872-56dd-450f-b295-e44224f1bd35 service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:51.762276 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472c4872-56dd-450f-b295-e44224f1bd35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:51.764022 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-472c4872-56dd-450f-b295-e44224f1bd35 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:13:51.773838 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472c4872-56dd-450f-b295-e44224f1bd35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:51.775252 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472c4872-56dd-450f-b295-e44224f1bd35 service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:51.783332 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1397/2812] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:13:51 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:13:52.521938 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511b58cc-1c0b-47c3-b034-0f1c9a9bea64 service nova] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:52.538479 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511b58cc-1c0b-47c3-b034-0f1c9a9bea64 service nova] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:52.545065 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72582532-51f9-4a9f-b226-ab3bd24cbdab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bd6caa461e84100a3d5fe5af5d80f3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:52.545984 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72582532-51f9-4a9f-b226-ab3bd24cbdab service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:52.546271 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72582532-51f9-4a9f-b226-ab3bd24cbdab service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:52.546711 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72582532-51f9-4a9f-b226-ab3bd24cbdab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:52.549011 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-72582532-51f9-4a9f-b226-ab3bd24cbdab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:13:52.562360 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72582532-51f9-4a9f-b226-ab3bd24cbdab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:52.563975 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72582532-51f9-4a9f-b226-ab3bd24cbdab service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:52.576407 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1416/2813] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:13:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:13:53.242973 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21014bb4-6c9b-42f0-8244-730d552354c9 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:53.243468 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21014bb4-6c9b-42f0-8244-730d552354c9 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:53.243566 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21014bb4-6c9b-42f0-8244-730d552354c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:53.261746 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-21014bb4-6c9b-42f0-8244-730d552354c9 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:13:53.262927 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-21014bb4-6c9b-42f0-8244-730d552354c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:13:53.270715 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1398/2814] 10.222.0.36 () {58 vars in 1016 bytes} [Fri Jun 10 13:13:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:13:53.290664 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72582532-51f9-4a9f-b226-ab3bd24cbdab service nova] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:53.301776 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72582532-51f9-4a9f-b226-ab3bd24cbdab service nova] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:53.307305 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5037694-9d66-4a83-ac8a-bc110cb3b13b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bd6caa461e84100a3d5fe5af5d80f3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:53.307940 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5037694-9d66-4a83-ac8a-bc110cb3b13b service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:53.311010 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5037694-9d66-4a83-ac8a-bc110cb3b13b service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:53.311010 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5037694-9d66-4a83-ac8a-bc110cb3b13b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:53.316630 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5037694-9d66-4a83-ac8a-bc110cb3b13b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:53.318222 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5037694-9d66-4a83-ac8a-bc110cb3b13b service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:53.327994 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1417/2815] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:13:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:13:58.603198 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21014bb4-6c9b-42f0-8244-730d552354c9 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:58.604791 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-21014bb4-6c9b-42f0-8244-730d552354c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:13:58.616962 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21014bb4-6c9b-42f0-8244-730d552354c9 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:58.621592 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e131410-03e0-472c-af62-92405283d1f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:58.622453 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e131410-03e0-472c-af62-92405283d1f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:58.622733 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e131410-03e0-472c-af62-92405283d1f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:58.623005 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e131410-03e0-472c-af62-92405283d1f3 admin admin] PATH_INFO: `/v3/users/e35f796f95b342ab9e2f514e62a4e44b` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:58.624220 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e131410-03e0-472c-af62-92405283d1f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e35f796f95b342ab9e2f514e62a4e44b)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:58.626202 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e131410-03e0-472c-af62-92405283d1f3 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:58.794339 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-9e131410-03e0-472c-af62-92405283d1f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e35f796f95b342ab9e2f514e62a4e44b'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:58.801045 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-9e131410-03e0-472c-af62-92405283d1f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e35f796f95b342ab9e2f514e62a4e44b'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:58.889467 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-9e131410-03e0-472c-af62-92405283d1f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e35f796f95b342ab9e2f514e62a4e44b'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:58.890079 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-9e131410-03e0-472c-af62-92405283d1f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e35f796f95b342ab9e2f514e62a4e44b'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:58.902760 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-9e131410-03e0-472c-af62-92405283d1f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e35f796f95b342ab9e2f514e62a4e44b'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:58.909020 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1399/2816] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:13:58 2022] DELETE /identity/v3/users/e35f796f95b342ab9e2f514e62a4e44b => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:13:58.913499 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f18562-8f8a-4c26-9f49-24e6b6b668fc None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:58.913979 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f18562-8f8a-4c26-9f49-24e6b6b668fc None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:58.914087 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f18562-8f8a-4c26-9f49-24e6b6b668fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:58.931833 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-b4f18562-8f8a-4c26-9f49-24e6b6b668fc None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:13:58.933209 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-b4f18562-8f8a-4c26-9f49-24e6b6b668fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:13:58.972830 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=100 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:13:58.991325 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1418/2817] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:13:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:13:59.001569 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e131410-03e0-472c-af62-92405283d1f3 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:59.003259 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-9e131410-03e0-472c-af62-92405283d1f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:13:59.071607 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e131410-03e0-472c-af62-92405283d1f3 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:59.071607 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5f2c09-64b3-4ea5-a10f-fec57d504e1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:59.072063 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5f2c09-64b3-4ea5-a10f-fec57d504e1c service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:59.072345 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5f2c09-64b3-4ea5-a10f-fec57d504e1c service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:59.072619 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5f2c09-64b3-4ea5-a10f-fec57d504e1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:59.082977 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5f2c09-64b3-4ea5-a10f-fec57d504e1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:59.084855 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5f2c09-64b3-4ea5-a10f-fec57d504e1c service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:59.097327 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1400/2818] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:13:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:13:59.215047 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b888bfa-2680-4846-9511-072f5cae35b4 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:59.215357 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b888bfa-2680-4846-9511-072f5cae35b4 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:59.215619 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b888bfa-2680-4846-9511-072f5cae35b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:59.234241 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-0b888bfa-2680-4846-9511-072f5cae35b4 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:13:59.235422 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0b888bfa-2680-4846-9511-072f5cae35b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:13:59.244047 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1419/2819] 10.222.0.36 () {58 vars in 1016 bytes} [Fri Jun 10 13:13:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:13:59.264685 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5f2c09-64b3-4ea5-a10f-fec57d504e1c service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:59.266616 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-ff5f2c09-64b3-4ea5-a10f-fec57d504e1c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:13:59.323385 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:13:59.354887 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5f2c09-64b3-4ea5-a10f-fec57d504e1c service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:59.361277 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec92ac4-8323-464c-8df6-07a0149e46fa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bd6caa461e84100a3d5fe5af5d80f3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:59.362538 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec92ac4-8323-464c-8df6-07a0149e46fa service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:59.362813 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec92ac4-8323-464c-8df6-07a0149e46fa service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:59.363148 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec92ac4-8323-464c-8df6-07a0149e46fa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:59.375952 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec92ac4-8323-464c-8df6-07a0149e46fa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:59.377577 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec92ac4-8323-464c-8df6-07a0149e46fa service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:59.391842 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1401/2820] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:13:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:13:59.568331 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b888bfa-2680-4846-9511-072f5cae35b4 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:59.568517 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0b888bfa-2680-4846-9511-072f5cae35b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:13:59.582098 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b888bfa-2680-4846-9511-072f5cae35b4 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:59.589075 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13eb2dd-0c91-4f2b-a4c3-290b17b2d7c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:59.596680 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13eb2dd-0c91-4f2b-a4c3-290b17b2d7c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:59.596680 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13eb2dd-0c91-4f2b-a4c3-290b17b2d7c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:59.596680 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13eb2dd-0c91-4f2b-a4c3-290b17b2d7c5 admin admin] PATH_INFO: `/v3/projects/46e74cb9018c4bd8904f84fc943f1a18` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:59.596680 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13eb2dd-0c91-4f2b-a4c3-290b17b2d7c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46e74cb9018c4bd8904f84fc943f1a18)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:59.596680 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13eb2dd-0c91-4f2b-a4c3-290b17b2d7c5 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:59.702738 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c13eb2dd-0c91-4f2b-a4c3-290b17b2d7c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46e74cb9018c4bd8904f84fc943f1a18', 'request_id': 'req-c13eb2dd-0c91-4f2b-a4c3-290b17b2d7c5'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:59.708468 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c13eb2dd-0c91-4f2b-a4c3-290b17b2d7c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46e74cb9018c4bd8904f84fc943f1a18', 'request_id': 'req-c13eb2dd-0c91-4f2b-a4c3-290b17b2d7c5'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:59.715479 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c13eb2dd-0c91-4f2b-a4c3-290b17b2d7c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46e74cb9018c4bd8904f84fc943f1a18', 'request_id': 'req-c13eb2dd-0c91-4f2b-a4c3-290b17b2d7c5'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:59.775169 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c13eb2dd-0c91-4f2b-a4c3-290b17b2d7c5 admin admin] The token cache is being invalidate because project 46e74cb9018c4bd8904f84fc943f1a18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:13:59.775580 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c13eb2dd-0c91-4f2b-a4c3-290b17b2d7c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:59.776135 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-c13eb2dd-0c91-4f2b-a4c3-290b17b2d7c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:13:59.777743 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1420/2821] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:13:59 2022] DELETE /identity/v3/projects/46e74cb9018c4bd8904f84fc943f1a18 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:13:59.783022 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c313df01-c8d0-4ab2-89d7-13c2c1a31792 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:59.783517 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c313df01-c8d0-4ab2-89d7-13c2c1a31792 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:59.783627 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c313df01-c8d0-4ab2-89d7-13c2c1a31792 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:59.802020 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-c313df01-c8d0-4ab2-89d7-13c2c1a31792 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:13:59.803220 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-c313df01-c8d0-4ab2-89d7-13c2c1a31792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:13:59.861906 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1402/2822] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:13:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:13:59.869238 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174f9a5f-9266-40fc-a437-9c42a56de271 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:59.869527 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174f9a5f-9266-40fc-a437-9c42a56de271 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:59.869805 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174f9a5f-9266-40fc-a437-9c42a56de271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:59.888423 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-174f9a5f-9266-40fc-a437-9c42a56de271 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:13:59.889604 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-174f9a5f-9266-40fc-a437-9c42a56de271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:13:59.897331 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1421/2823] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:13:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:13:59.906903 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c313df01-c8d0-4ab2-89d7-13c2c1a31792 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:59.912184 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c313df01-c8d0-4ab2-89d7-13c2c1a31792 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:59.916808 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455a5f4e-b525-477c-a2d0-17be0f3ac117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:59.917485 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455a5f4e-b525-477c-a2d0-17be0f3ac117 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:59.917769 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455a5f4e-b525-477c-a2d0-17be0f3ac117 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:59.918042 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455a5f4e-b525-477c-a2d0-17be0f3ac117 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:59.918767 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455a5f4e-b525-477c-a2d0-17be0f3ac117 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 13:13:59.919073 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455a5f4e-b525-477c-a2d0-17be0f3ac117 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:59.920696 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455a5f4e-b525-477c-a2d0-17be0f3ac117 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:13:59.932351 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1403/2824] 10.222.0.27 () {58 vars in 1152 bytes} [Fri Jun 10 13:13:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 13:13:59.936639 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174f9a5f-9266-40fc-a437-9c42a56de271 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:13:59.946357 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174f9a5f-9266-40fc-a437-9c42a56de271 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:13:59.951084 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82820182-44ea-4397-a1aa-2329dafedb1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:13:59.951734 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82820182-44ea-4397-a1aa-2329dafedb1d admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:13:59.952058 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82820182-44ea-4397-a1aa-2329dafedb1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:13:59.952341 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82820182-44ea-4397-a1aa-2329dafedb1d admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:13:59.953040 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82820182-44ea-4397-a1aa-2329dafedb1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:13:59.954922 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82820182-44ea-4397-a1aa-2329dafedb1d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:14:00.021730 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1422/2825] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:13:59 2022] POST /identity/v3/projects => generated 369 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 10 13:14:00.026463 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455a5f4e-b525-477c-a2d0-17be0f3ac117 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:14:00.036926 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455a5f4e-b525-477c-a2d0-17be0f3ac117 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:14:00.092084 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1e3a9b-2820-4b5d-ac94-91e41e10f4a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:14:00.092744 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1e3a9b-2820-4b5d-ac94-91e41e10f4a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:14:00.093028 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1e3a9b-2820-4b5d-ac94-91e41e10f4a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:14:00.093306 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1e3a9b-2820-4b5d-ac94-91e41e10f4a4 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:14:00.093979 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1e3a9b-2820-4b5d-ac94-91e41e10f4a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:14:00.095891 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1e3a9b-2820-4b5d-ac94-91e41e10f4a4 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:14:00.171358 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1404/2826] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:14:00 2022] POST /identity/v3/users => generated 285 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:14:00.175309 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82820182-44ea-4397-a1aa-2329dafedb1d admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:14:00.185569 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82820182-44ea-4397-a1aa-2329dafedb1d admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:14:00.190185 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdf37c5-86cc-4d0c-b97a-bfda1abd7828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:14:00.190847 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdf37c5-86cc-4d0c-b97a-bfda1abd7828 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:14:00.191303 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdf37c5-86cc-4d0c-b97a-bfda1abd7828 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:14:00.191588 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdf37c5-86cc-4d0c-b97a-bfda1abd7828 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:14:00.192367 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdf37c5-86cc-4d0c-b97a-bfda1abd7828 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:14:00.193932 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdf37c5-86cc-4d0c-b97a-bfda1abd7828 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:14:00.212451 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1423/2827] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:14:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:14:00.213371 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1e3a9b-2820-4b5d-ac94-91e41e10f4a4 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:14:00.233219 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1e3a9b-2820-4b5d-ac94-91e41e10f4a4 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:14:00.238215 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ad9edb-a9c9-49f1-8053-4f5f357c51a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:14:00.238906 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ad9edb-a9c9-49f1-8053-4f5f357c51a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:14:00.239197 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ad9edb-a9c9-49f1-8053-4f5f357c51a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:14:00.239484 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ad9edb-a9c9-49f1-8053-4f5f357c51a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:14:00.240265 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ad9edb-a9c9-49f1-8053-4f5f357c51a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:14:00.241904 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ad9edb-a9c9-49f1-8053-4f5f357c51a0 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:14:00.255780 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1405/2828] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:14:00 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 10 13:14:00.257283 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdf37c5-86cc-4d0c-b97a-bfda1abd7828 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:14:00.267794 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdf37c5-86cc-4d0c-b97a-bfda1abd7828 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:14:00.272462 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee28a344-ea50-483e-8be1-41ea4131d0a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:14:00.273343 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee28a344-ea50-483e-8be1-41ea4131d0a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:14:00.273627 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee28a344-ea50-483e-8be1-41ea4131d0a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:14:00.274493 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee28a344-ea50-483e-8be1-41ea4131d0a6 admin admin] PATH_INFO: `/v3/projects/b2a9d8c3c38f4a5d9751b49128d86b72/users/db9c896d788b419c97b571b75da897b6/roles/43106c664e1b40de984c0c62701149c8` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:14:00.290593 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee28a344-ea50-483e-8be1-41ea4131d0a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2a9d8c3c38f4a5d9751b49128d86b72, user_id=db9c896d788b419c97b571b75da897b6, role_id=43106c664e1b40de984c0c62701149c8)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:14:00.293262 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee28a344-ea50-483e-8be1-41ea4131d0a6 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:14:00.393127 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1424/2829] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:14:00 2022] PUT /identity/v3/projects/b2a9d8c3c38f4a5d9751b49128d86b72/users/db9c896d788b419c97b571b75da897b6/roles/43106c664e1b40de984c0c62701149c8 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:14:00.403235 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccb8f1e-6dd8-4b58-9109-65d375cc86f8 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:14:00.403526 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccb8f1e-6dd8-4b58-9109-65d375cc86f8 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:14:00.403807 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccb8f1e-6dd8-4b58-9109-65d375cc86f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:14:00.431104 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-0ccb8f1e-6dd8-4b58-9109-65d375cc86f8 None None] MFA Rules not processed for user `db9c896d788b419c97b571b75da897b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:14:00.432272 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-0ccb8f1e-6dd8-4b58-9109-65d375cc86f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:14:00.456555 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:14:00.497758 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1406/2830] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:14:00 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:14:00.510381 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee28a344-ea50-483e-8be1-41ea4131d0a6 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:14:00.512392 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ee28a344-ea50-483e-8be1-41ea4131d0a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:14:00.571238 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee28a344-ea50-483e-8be1-41ea4131d0a6 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:14:00.574971 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1869e20f-f473-4b01-b7e8-566e909245fd service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6d3eb01a98274050b930616b98998c06', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:14:00.575624 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1869e20f-f473-4b01-b7e8-566e909245fd service glance] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:14:00.576025 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1869e20f-f473-4b01-b7e8-566e909245fd service glance] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:14:00.576228 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1869e20f-f473-4b01-b7e8-566e909245fd service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:14:00.585147 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1869e20f-f473-4b01-b7e8-566e909245fd service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:14:00.586580 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1869e20f-f473-4b01-b7e8-566e909245fd service glance] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:14:00.591726 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:14:00.596414 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1425/2831] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:14:00 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 10 13:14:03.461398 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10a2ca0-faf8-4cb8-af44-1db758ea8d1a None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:14:03.461881 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10a2ca0-faf8-4cb8-af44-1db758ea8d1a None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:14:03.461986 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10a2ca0-faf8-4cb8-af44-1db758ea8d1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:14:03.480954 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-a10a2ca0-faf8-4cb8-af44-1db758ea8d1a None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:14:03.486108 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a10a2ca0-faf8-4cb8-af44-1db758ea8d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:14:03.541327 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1407/2832] 10.222.0.36 () {58 vars in 1016 bytes} [Fri Jun 10 13:14:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:14:03.563268 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1869e20f-f473-4b01-b7e8-566e909245fd service glance] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:14:03.564691 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-1869e20f-f473-4b01-b7e8-566e909245fd service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:14:03.639182 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1869e20f-f473-4b01-b7e8-566e909245fd service glance] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:14:03.644698 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93424059-2947-4b95-bfbd-caa6bb6e0801 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bd6caa461e84100a3d5fe5af5d80f3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:14:03.645354 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93424059-2947-4b95-bfbd-caa6bb6e0801 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:14:03.645646 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93424059-2947-4b95-bfbd-caa6bb6e0801 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:14:03.645933 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93424059-2947-4b95-bfbd-caa6bb6e0801 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:14:03.654928 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93424059-2947-4b95-bfbd-caa6bb6e0801 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:14:03.656821 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93424059-2947-4b95-bfbd-caa6bb6e0801 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:14:03.666729 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1426/2833] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:14:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:14:05.993910 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10a2ca0-faf8-4cb8-af44-1db758ea8d1a None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:14:05.995389 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a10a2ca0-faf8-4cb8-af44-1db758ea8d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:14:06.052942 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10a2ca0-faf8-4cb8-af44-1db758ea8d1a None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:14:06.057128 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bea87f-19c6-45aa-aa44-d13db042156d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6cc95a563b941ac92ab283aa0d5a5d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:14:06.057778 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bea87f-19c6-45aa-aa44-d13db042156d service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:14:06.058065 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bea87f-19c6-45aa-aa44-d13db042156d service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:14:06.058347 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bea87f-19c6-45aa-aa44-d13db042156d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:14:06.060323 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-59bea87f-19c6-45aa-aa44-d13db042156d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:14:06.125381 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bea87f-19c6-45aa-aa44-d13db042156d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:14:06.126744 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bea87f-19c6-45aa-aa44-d13db042156d service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:14:06.140036 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1408/2834] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:14:05 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:14:07.699795 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ce973b-e43e-4cd5-be1c-d129c5127531 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:14:07.700365 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ce973b-e43e-4cd5-be1c-d129c5127531 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:14:07.700365 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ce973b-e43e-4cd5-be1c-d129c5127531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:14:07.717623 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-85ce973b-e43e-4cd5-be1c-d129c5127531 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:14:07.718845 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-85ce973b-e43e-4cd5-be1c-d129c5127531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:14:07.726692 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1427/2835] 10.222.0.36 () {58 vars in 1016 bytes} [Fri Jun 10 13:14:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:14:07.749536 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bea87f-19c6-45aa-aa44-d13db042156d service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:14:07.761696 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bea87f-19c6-45aa-aa44-d13db042156d service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:14:07.767018 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596172f5-1ee2-4d60-ae8a-f69f3641f222 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bd6caa461e84100a3d5fe5af5d80f3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:14:07.767646 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596172f5-1ee2-4d60-ae8a-f69f3641f222 service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:14:07.767961 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596172f5-1ee2-4d60-ae8a-f69f3641f222 service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:14:07.768243 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596172f5-1ee2-4d60-ae8a-f69f3641f222 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:14:07.776869 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596172f5-1ee2-4d60-ae8a-f69f3641f222 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:14:07.778381 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596172f5-1ee2-4d60-ae8a-f69f3641f222 service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:14:07.788114 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1409/2836] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:14:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:14:17.712938 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=37 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:14:17.783728 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:14:26.026049 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845a8d97-a641-466d-97af-81b26e1ef0df None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:14:26.030690 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845a8d97-a641-466d-97af-81b26e1ef0df None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:14:26.030690 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845a8d97-a641-466d-97af-81b26e1ef0df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:14:26.045336 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-845a8d97-a641-466d-97af-81b26e1ef0df None None] MFA Rules not processed for user `7db0e455e2c841b5ae1e9b60f3dd0906`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:14:26.046519 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-845a8d97-a641-466d-97af-81b26e1ef0df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:14:26.100930 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1428/2837] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:14:26 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:14:31.893555 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596172f5-1ee2-4d60-ae8a-f69f3641f222 service nova] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:14:31.903625 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596172f5-1ee2-4d60-ae8a-f69f3641f222 service nova] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:14:31.908393 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904fcd07-d8a5-4f75-973f-d0b5baab9dd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:14:31.909244 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904fcd07-d8a5-4f75-973f-d0b5baab9dd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:14:31.909527 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904fcd07-d8a5-4f75-973f-d0b5baab9dd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:14:31.909803 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904fcd07-d8a5-4f75-973f-d0b5baab9dd3 admin admin] PATH_INFO: `/v3/users/db9c896d788b419c97b571b75da897b6` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:14:31.910970 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904fcd07-d8a5-4f75-973f-d0b5baab9dd3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db9c896d788b419c97b571b75da897b6)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:14:31.912926 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904fcd07-d8a5-4f75-973f-d0b5baab9dd3 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:14:32.067662 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-904fcd07-d8a5-4f75-973f-d0b5baab9dd3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db9c896d788b419c97b571b75da897b6'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:14:32.074937 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-904fcd07-d8a5-4f75-973f-d0b5baab9dd3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db9c896d788b419c97b571b75da897b6'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:14:32.134905 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-904fcd07-d8a5-4f75-973f-d0b5baab9dd3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db9c896d788b419c97b571b75da897b6'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:14:32.137437 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-904fcd07-d8a5-4f75-973f-d0b5baab9dd3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db9c896d788b419c97b571b75da897b6'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:14:32.149656 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-904fcd07-d8a5-4f75-973f-d0b5baab9dd3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db9c896d788b419c97b571b75da897b6'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:14:32.156758 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1410/2838] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:14:31 2022] DELETE /identity/v3/users/db9c896d788b419c97b571b75da897b6 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:14:32.161333 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0727aa8c-2986-453f-abe7-01008be30b0f None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:14:32.161917 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0727aa8c-2986-453f-abe7-01008be30b0f None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:14:32.161917 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0727aa8c-2986-453f-abe7-01008be30b0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:14:32.180157 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-0727aa8c-2986-453f-abe7-01008be30b0f None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:14:32.181383 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0727aa8c-2986-453f-abe7-01008be30b0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:14:32.241682 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1429/2839] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:14:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:14:32.251606 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904fcd07-d8a5-4f75-973f-d0b5baab9dd3 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:14:32.253078 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-904fcd07-d8a5-4f75-973f-d0b5baab9dd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:14:32.313382 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904fcd07-d8a5-4f75-973f-d0b5baab9dd3 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:14:32.317023 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4771ec8-f410-4b11-9ec5-6af140b5f1fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:14:32.317667 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4771ec8-f410-4b11-9ec5-6af140b5f1fd service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:14:32.317947 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4771ec8-f410-4b11-9ec5-6af140b5f1fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:14:32.318223 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4771ec8-f410-4b11-9ec5-6af140b5f1fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:14:32.328055 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4771ec8-f410-4b11-9ec5-6af140b5f1fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:14:32.329510 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4771ec8-f410-4b11-9ec5-6af140b5f1fd service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:14:32.340011 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1411/2840] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:14:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:14:32.915565 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0727aa8c-2986-453f-abe7-01008be30b0f None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:14:32.915565 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0727aa8c-2986-453f-abe7-01008be30b0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:14:32.939252 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0727aa8c-2986-453f-abe7-01008be30b0f None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:14:32.947780 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da0de9b-67fe-4cf4-9055-292a80d52f4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:14:32.947780 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da0de9b-67fe-4cf4-9055-292a80d52f4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:14:32.947780 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da0de9b-67fe-4cf4-9055-292a80d52f4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:14:32.947780 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da0de9b-67fe-4cf4-9055-292a80d52f4b admin admin] PATH_INFO: `/v3/projects/b2a9d8c3c38f4a5d9751b49128d86b72` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:14:32.948651 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da0de9b-67fe-4cf4-9055-292a80d52f4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2a9d8c3c38f4a5d9751b49128d86b72)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:14:32.949984 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da0de9b-67fe-4cf4-9055-292a80d52f4b admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:14:33.045734 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-7da0de9b-67fe-4cf4-9055-292a80d52f4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2a9d8c3c38f4a5d9751b49128d86b72', 'request_id': 'req-7da0de9b-67fe-4cf4-9055-292a80d52f4b'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:14:33.051303 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-7da0de9b-67fe-4cf4-9055-292a80d52f4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2a9d8c3c38f4a5d9751b49128d86b72', 'request_id': 'req-7da0de9b-67fe-4cf4-9055-292a80d52f4b'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:14:33.057489 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-7da0de9b-67fe-4cf4-9055-292a80d52f4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2a9d8c3c38f4a5d9751b49128d86b72', 'request_id': 'req-7da0de9b-67fe-4cf4-9055-292a80d52f4b'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:14:33.117312 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-7da0de9b-67fe-4cf4-9055-292a80d52f4b admin admin] The token cache is being invalidate because project b2a9d8c3c38f4a5d9751b49128d86b72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:14:33.117604 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-7da0de9b-67fe-4cf4-9055-292a80d52f4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:14:33.118231 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-7da0de9b-67fe-4cf4-9055-292a80d52f4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:14:33.119774 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1430/2841] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:14:32 2022] DELETE /identity/v3/projects/b2a9d8c3c38f4a5d9751b49128d86b72 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:14:33.495053 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4771ec8-f410-4b11-9ec5-6af140b5f1fd service neutron] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:14:33.496668 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-d4771ec8-f410-4b11-9ec5-6af140b5f1fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:14:33.509806 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4771ec8-f410-4b11-9ec5-6af140b5f1fd service neutron] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:14:33.566850 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c03774-f284-462e-a0cf-d7fcb11739cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:14:33.567898 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c03774-f284-462e-a0cf-d7fcb11739cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:14:33.568111 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c03774-f284-462e-a0cf-d7fcb11739cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:14:33.568382 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c03774-f284-462e-a0cf-d7fcb11739cc admin admin] PATH_INFO: `/v3/users/8b04b18237334532afda8607312c973f` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:14:33.569576 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c03774-f284-462e-a0cf-d7fcb11739cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b04b18237334532afda8607312c973f)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:14:33.571477 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c03774-f284-462e-a0cf-d7fcb11739cc admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:14:33.588025 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:14:33.766795 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f8c03774-f284-462e-a0cf-d7fcb11739cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b04b18237334532afda8607312c973f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:14:33.773634 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f8c03774-f284-462e-a0cf-d7fcb11739cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b04b18237334532afda8607312c973f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:14:33.831644 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f8c03774-f284-462e-a0cf-d7fcb11739cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b04b18237334532afda8607312c973f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:14:33.832283 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f8c03774-f284-462e-a0cf-d7fcb11739cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b04b18237334532afda8607312c973f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:14:33.845103 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f8c03774-f284-462e-a0cf-d7fcb11739cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b04b18237334532afda8607312c973f'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:14:33.851056 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1412/2842] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:14:33 2022] DELETE /identity/v3/users/8b04b18237334532afda8607312c973f => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:14:34.067577 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da0de9b-67fe-4cf4-9055-292a80d52f4b admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:14:34.069301 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-7da0de9b-67fe-4cf4-9055-292a80d52f4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:14:34.082876 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da0de9b-67fe-4cf4-9055-292a80d52f4b admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:14:34.108793 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:14:34.139238 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b37b5ea-5996-47b5-9ba6-7537634f9a11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:14:34.140118 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b37b5ea-5996-47b5-9ba6-7537634f9a11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:14:34.140403 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b37b5ea-5996-47b5-9ba6-7537634f9a11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:14:34.140681 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b37b5ea-5996-47b5-9ba6-7537634f9a11 admin admin] PATH_INFO: `/v3/projects/94ba1ecd5b9e49108a46d7064122de04` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:14:34.141850 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b37b5ea-5996-47b5-9ba6-7537634f9a11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=94ba1ecd5b9e49108a46d7064122de04)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:14:34.143789 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b37b5ea-5996-47b5-9ba6-7537634f9a11 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:14:34.241949 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6b37b5ea-5996-47b5-9ba6-7537634f9a11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94ba1ecd5b9e49108a46d7064122de04', 'request_id': 'req-6b37b5ea-5996-47b5-9ba6-7537634f9a11'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:14:34.248145 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6b37b5ea-5996-47b5-9ba6-7537634f9a11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94ba1ecd5b9e49108a46d7064122de04', 'request_id': 'req-6b37b5ea-5996-47b5-9ba6-7537634f9a11'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:14:34.254139 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6b37b5ea-5996-47b5-9ba6-7537634f9a11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94ba1ecd5b9e49108a46d7064122de04', 'request_id': 'req-6b37b5ea-5996-47b5-9ba6-7537634f9a11'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:14:34.338559 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6b37b5ea-5996-47b5-9ba6-7537634f9a11 admin admin] The token cache is being invalidate because project 94ba1ecd5b9e49108a46d7064122de04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:14:34.338915 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6b37b5ea-5996-47b5-9ba6-7537634f9a11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:14:34.339537 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-6b37b5ea-5996-47b5-9ba6-7537634f9a11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:14:34.341224 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1431/2843] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:14:34 2022] DELETE /identity/v3/projects/94ba1ecd5b9e49108a46d7064122de04 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:14:34.345273 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c03774-f284-462e-a0cf-d7fcb11739cc admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:14:34.346685 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f8c03774-f284-462e-a0cf-d7fcb11739cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:14:34.359785 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c03774-f284-462e-a0cf-d7fcb11739cc admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:14:34.415617 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e20a9da-b642-4e10-9817-bb4810750c7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:14:34.416527 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e20a9da-b642-4e10-9817-bb4810750c7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:14:34.416807 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e20a9da-b642-4e10-9817-bb4810750c7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:14:34.417081 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e20a9da-b642-4e10-9817-bb4810750c7c admin admin] PATH_INFO: `/v3/users/e891db98760447cda84a31c715a52234` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:14:34.418746 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e20a9da-b642-4e10-9817-bb4810750c7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e891db98760447cda84a31c715a52234)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:14:34.420937 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e20a9da-b642-4e10-9817-bb4810750c7c admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:14:34.587671 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6e20a9da-b642-4e10-9817-bb4810750c7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e891db98760447cda84a31c715a52234'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:14:34.594472 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6e20a9da-b642-4e10-9817-bb4810750c7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e891db98760447cda84a31c715a52234'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:14:34.660458 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6e20a9da-b642-4e10-9817-bb4810750c7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e891db98760447cda84a31c715a52234'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:14:34.661071 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6e20a9da-b642-4e10-9817-bb4810750c7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e891db98760447cda84a31c715a52234'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:14:34.673756 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-6e20a9da-b642-4e10-9817-bb4810750c7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e891db98760447cda84a31c715a52234'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:14:34.679596 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1413/2844] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:14:34 2022] DELETE /identity/v3/users/e891db98760447cda84a31c715a52234 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 10 13:14:35.010107 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b37b5ea-5996-47b5-9ba6-7537634f9a11 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:14:35.011526 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-6b37b5ea-5996-47b5-9ba6-7537634f9a11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:14:35.026628 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b37b5ea-5996-47b5-9ba6-7537634f9a11 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:14:35.085578 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94685ab4-4812-4985-86f4-d052cef462a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:14:35.086437 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94685ab4-4812-4985-86f4-d052cef462a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:14:35.086722 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94685ab4-4812-4985-86f4-d052cef462a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:14:35.087002 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94685ab4-4812-4985-86f4-d052cef462a0 admin admin] PATH_INFO: `/v3/projects/e3d144b60d1042958ff751b86bf6c53b` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:14:35.088277 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94685ab4-4812-4985-86f4-d052cef462a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3d144b60d1042958ff751b86bf6c53b)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:14:35.090818 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94685ab4-4812-4985-86f4-d052cef462a0 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:14:35.246889 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-94685ab4-4812-4985-86f4-d052cef462a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3d144b60d1042958ff751b86bf6c53b', 'request_id': 'req-94685ab4-4812-4985-86f4-d052cef462a0'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:14:35.262687 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-94685ab4-4812-4985-86f4-d052cef462a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3d144b60d1042958ff751b86bf6c53b', 'request_id': 'req-94685ab4-4812-4985-86f4-d052cef462a0'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:14:35.264689 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:14:35.270997 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-94685ab4-4812-4985-86f4-d052cef462a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3d144b60d1042958ff751b86bf6c53b', 'request_id': 'req-94685ab4-4812-4985-86f4-d052cef462a0'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:14:35.362544 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-94685ab4-4812-4985-86f4-d052cef462a0 admin admin] The token cache is being invalidate because project e3d144b60d1042958ff751b86bf6c53b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:14:35.362837 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-94685ab4-4812-4985-86f4-d052cef462a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:14:35.363485 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-94685ab4-4812-4985-86f4-d052cef462a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:14:35.365052 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1432/2845] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:14:35 2022] DELETE /identity/v3/projects/e3d144b60d1042958ff751b86bf6c53b => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:14:35.369900 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95ca377-e168-4f82-9672-277439d0774e None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:14:35.370198 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95ca377-e168-4f82-9672-277439d0774e None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:14:35.370482 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95ca377-e168-4f82-9672-277439d0774e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:14:35.388898 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-b95ca377-e168-4f82-9672-277439d0774e None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:14:35.390173 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-b95ca377-e168-4f82-9672-277439d0774e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:14:35.398514 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:DELETE=19,keystone:SELECT=79,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:14:35.448165 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1414/2846] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:14:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:14:35.457004 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89105d93-81d3-4ec8-9818-80712cd198a4 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:14:35.457297 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89105d93-81d3-4ec8-9818-80712cd198a4 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:14:35.457580 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89105d93-81d3-4ec8-9818-80712cd198a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:14:35.476731 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-89105d93-81d3-4ec8-9818-80712cd198a4 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:14:35.477910 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-89105d93-81d3-4ec8-9818-80712cd198a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:14:35.485730 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1433/2847] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:14:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:14:35.489859 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95ca377-e168-4f82-9672-277439d0774e None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:14:35.502729 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95ca377-e168-4f82-9672-277439d0774e None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:14:35.505015 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70be1ccb-582c-439c-92bc-e06b22f3bb5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:14:35.505667 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70be1ccb-582c-439c-92bc-e06b22f3bb5f admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:14:35.505950 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70be1ccb-582c-439c-92bc-e06b22f3bb5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:14:35.506226 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70be1ccb-582c-439c-92bc-e06b22f3bb5f admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:14:35.506945 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70be1ccb-582c-439c-92bc-e06b22f3bb5f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 13:14:35.507251 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70be1ccb-582c-439c-92bc-e06b22f3bb5f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:14:35.508975 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70be1ccb-582c-439c-92bc-e06b22f3bb5f admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:14:35.521451 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1415/2848] 10.222.0.27 () {58 vars in 1152 bytes} [Fri Jun 10 13:14:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 13:14:35.525727 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89105d93-81d3-4ec8-9818-80712cd198a4 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:14:35.535082 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89105d93-81d3-4ec8-9818-80712cd198a4 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:14:35.539734 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d1a2eb-bc32-4e61-9776-b6554fde9da0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:14:35.540417 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d1a2eb-bc32-4e61-9776-b6554fde9da0 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:14:35.540703 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d1a2eb-bc32-4e61-9776-b6554fde9da0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:14:35.540982 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d1a2eb-bc32-4e61-9776-b6554fde9da0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:14:35.541668 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d1a2eb-bc32-4e61-9776-b6554fde9da0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:14:35.543568 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d1a2eb-bc32-4e61-9776-b6554fde9da0 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:14:35.613794 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1434/2849] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:14:35 2022] POST /identity/v3/projects => generated 385 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 10 13:14:35.615078 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70be1ccb-582c-439c-92bc-e06b22f3bb5f admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:14:35.625618 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70be1ccb-582c-439c-92bc-e06b22f3bb5f admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:14:35.683050 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f62c775-2bb1-4ef0-a925-f33f4271cf47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:14:35.684320 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f62c775-2bb1-4ef0-a925-f33f4271cf47 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:14:35.684565 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f62c775-2bb1-4ef0-a925-f33f4271cf47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:14:35.684846 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f62c775-2bb1-4ef0-a925-f33f4271cf47 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:14:35.685543 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f62c775-2bb1-4ef0-a925-f33f4271cf47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:14:35.687505 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f62c775-2bb1-4ef0-a925-f33f4271cf47 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:14:35.778726 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1416/2850] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:14:35 2022] POST /identity/v3/users => generated 293 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:14:35.779878 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d1a2eb-bc32-4e61-9776-b6554fde9da0 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:14:35.791201 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d1a2eb-bc32-4e61-9776-b6554fde9da0 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:14:35.796440 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8bd298-77ff-4173-b0b8-0c789babddfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:14:35.797105 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8bd298-77ff-4173-b0b8-0c789babddfa admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:14:35.800717 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8bd298-77ff-4173-b0b8-0c789babddfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:14:35.800717 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8bd298-77ff-4173-b0b8-0c789babddfa admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:14:35.800717 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8bd298-77ff-4173-b0b8-0c789babddfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:14:35.800717 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8bd298-77ff-4173-b0b8-0c789babddfa admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:14:35.811519 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1435/2851] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:14:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:14:35.815784 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f62c775-2bb1-4ef0-a925-f33f4271cf47 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:14:35.826130 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f62c775-2bb1-4ef0-a925-f33f4271cf47 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:14:35.830952 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef20af0-9f70-48d8-a679-0a520ca89d20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:14:35.831683 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef20af0-9f70-48d8-a679-0a520ca89d20 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:14:35.832088 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef20af0-9f70-48d8-a679-0a520ca89d20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:14:35.832450 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef20af0-9f70-48d8-a679-0a520ca89d20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:14:35.833260 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef20af0-9f70-48d8-a679-0a520ca89d20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:14:35.834863 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef20af0-9f70-48d8-a679-0a520ca89d20 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:14:35.847269 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1417/2852] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:14:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:14:35.851362 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8bd298-77ff-4173-b0b8-0c789babddfa admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:14:35.862555 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8bd298-77ff-4173-b0b8-0c789babddfa admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:14:35.867334 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051af1e9-e330-4073-9a4e-29da56a8dfe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:14:35.868235 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051af1e9-e330-4073-9a4e-29da56a8dfe9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:14:35.868512 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051af1e9-e330-4073-9a4e-29da56a8dfe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:14:35.868788 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051af1e9-e330-4073-9a4e-29da56a8dfe9 admin admin] PATH_INFO: `/v3/projects/963a3d7d5c0143a299527f020e7a37a6/users/d242d51eee804f4aa557607342bb3564/roles/43106c664e1b40de984c0c62701149c8` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:14:35.884193 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051af1e9-e330-4073-9a4e-29da56a8dfe9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=963a3d7d5c0143a299527f020e7a37a6, user_id=d242d51eee804f4aa557607342bb3564, role_id=43106c664e1b40de984c0c62701149c8)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:14:35.886808 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051af1e9-e330-4073-9a4e-29da56a8dfe9 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:14:35.959080 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1436/2853] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:14:35 2022] PUT /identity/v3/projects/963a3d7d5c0143a299527f020e7a37a6/users/d242d51eee804f4aa557607342bb3564/roles/43106c664e1b40de984c0c62701149c8 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 10 13:14:35.969137 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d115350-f89f-4f89-9cae-2c0a067c38ab None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:14:35.969440 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d115350-f89f-4f89-9cae-2c0a067c38ab None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:14:35.969725 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d115350-f89f-4f89-9cae-2c0a067c38ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:14:35.998320 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-9d115350-f89f-4f89-9cae-2c0a067c38ab None None] MFA Rules not processed for user `d242d51eee804f4aa557607342bb3564`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:14:35.999474 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-9d115350-f89f-4f89-9cae-2c0a067c38ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:14:36.064757 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1418/2854] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:14:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:14:36.076268 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051af1e9-e330-4073-9a4e-29da56a8dfe9 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:14:36.078158 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-051af1e9-e330-4073-9a4e-29da56a8dfe9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:14:36.135252 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051af1e9-e330-4073-9a4e-29da56a8dfe9 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:14:36.139028 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c952e8d4-3c29-40b8-b2c9-e26c327bed4a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6cc95a563b941ac92ab283aa0d5a5d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:14:36.139670 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c952e8d4-3c29-40b8-b2c9-e26c327bed4a service cinder] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:14:36.140031 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c952e8d4-3c29-40b8-b2c9-e26c327bed4a service cinder] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:14:36.145968 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c952e8d4-3c29-40b8-b2c9-e26c327bed4a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:14:36.148805 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c952e8d4-3c29-40b8-b2c9-e26c327bed4a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:14:36.150217 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c952e8d4-3c29-40b8-b2c9-e26c327bed4a service cinder] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:14:36.159851 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1437/2855] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:14:36 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:14:36.179165 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d115350-f89f-4f89-9cae-2c0a067c38ab None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:14:36.188897 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:14:36.242200 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d115350-f89f-4f89-9cae-2c0a067c38ab None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:14:36.246726 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0125a8c-91d4-4851-a05a-efbd1478bb18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:14:36.247591 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0125a8c-91d4-4851-a05a-efbd1478bb18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:14:36.247888 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0125a8c-91d4-4851-a05a-efbd1478bb18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:14:36.248332 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0125a8c-91d4-4851-a05a-efbd1478bb18 admin admin] PATH_INFO: `/v3/users/d242d51eee804f4aa557607342bb3564` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:14:36.249547 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0125a8c-91d4-4851-a05a-efbd1478bb18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d242d51eee804f4aa557607342bb3564)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:14:36.251440 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0125a8c-91d4-4851-a05a-efbd1478bb18 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:14:36.395120 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f0125a8c-91d4-4851-a05a-efbd1478bb18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd242d51eee804f4aa557607342bb3564'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:14:36.402389 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f0125a8c-91d4-4851-a05a-efbd1478bb18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd242d51eee804f4aa557607342bb3564'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:14:36.472279 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f0125a8c-91d4-4851-a05a-efbd1478bb18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd242d51eee804f4aa557607342bb3564'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:14:36.472913 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f0125a8c-91d4-4851-a05a-efbd1478bb18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd242d51eee804f4aa557607342bb3564'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:14:36.485947 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f0125a8c-91d4-4851-a05a-efbd1478bb18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd242d51eee804f4aa557607342bb3564'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:14:36.492092 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1419/2856] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:14:36 2022] DELETE /identity/v3/users/d242d51eee804f4aa557607342bb3564 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:14:36.496607 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d268a18-fd2c-459c-a2a4-6ee285a1b7ee None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:14:36.496893 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d268a18-fd2c-459c-a2a4-6ee285a1b7ee None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:14:36.497174 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d268a18-fd2c-459c-a2a4-6ee285a1b7ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:14:36.515397 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-8d268a18-fd2c-459c-a2a4-6ee285a1b7ee None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:14:36.516684 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8d268a18-fd2c-459c-a2a4-6ee285a1b7ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:14:36.581829 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1438/2857] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:14:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:14:36.592756 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0125a8c-91d4-4851-a05a-efbd1478bb18 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:14:36.594128 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-f0125a8c-91d4-4851-a05a-efbd1478bb18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:14:36.650734 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0125a8c-91d4-4851-a05a-efbd1478bb18 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:14:36.654978 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df0f88c-ceaa-4d75-8934-6f9a45df831e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:14:36.655882 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df0f88c-ceaa-4d75-8934-6f9a45df831e service neutron] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:14:36.656173 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df0f88c-ceaa-4d75-8934-6f9a45df831e service neutron] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:14:36.656450 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df0f88c-ceaa-4d75-8934-6f9a45df831e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:14:36.665332 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df0f88c-ceaa-4d75-8934-6f9a45df831e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:14:36.666746 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df0f88c-ceaa-4d75-8934-6f9a45df831e service neutron] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:14:36.676114 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1420/2858] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:14:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:14:37.039259 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d268a18-fd2c-459c-a2a4-6ee285a1b7ee None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:14:37.040746 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8d268a18-fd2c-459c-a2a4-6ee285a1b7ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:14:37.054660 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d268a18-fd2c-459c-a2a4-6ee285a1b7ee None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:14:37.060183 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce590557-fbd0-4015-b858-b02ae4229aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:14:37.061509 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce590557-fbd0-4015-b858-b02ae4229aaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:14:37.062667 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce590557-fbd0-4015-b858-b02ae4229aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:14:37.062667 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce590557-fbd0-4015-b858-b02ae4229aaf admin admin] PATH_INFO: `/v3/projects/963a3d7d5c0143a299527f020e7a37a6` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:14:37.063745 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce590557-fbd0-4015-b858-b02ae4229aaf admin admin] RBAC: Authorizing `identity:delete_project(project_id=963a3d7d5c0143a299527f020e7a37a6)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:14:37.065672 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce590557-fbd0-4015-b858-b02ae4229aaf admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:14:37.091343 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:14:37.154552 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ce590557-fbd0-4015-b858-b02ae4229aaf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '963a3d7d5c0143a299527f020e7a37a6', 'request_id': 'req-ce590557-fbd0-4015-b858-b02ae4229aaf'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:14:37.160446 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ce590557-fbd0-4015-b858-b02ae4229aaf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '963a3d7d5c0143a299527f020e7a37a6', 'request_id': 'req-ce590557-fbd0-4015-b858-b02ae4229aaf'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:14:37.166750 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ce590557-fbd0-4015-b858-b02ae4229aaf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '963a3d7d5c0143a299527f020e7a37a6', 'request_id': 'req-ce590557-fbd0-4015-b858-b02ae4229aaf'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:14:37.231606 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ce590557-fbd0-4015-b858-b02ae4229aaf admin admin] The token cache is being invalidate because project 963a3d7d5c0143a299527f020e7a37a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80237) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:14:37.231924 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ce590557-fbd0-4015-b858-b02ae4229aaf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:14:37.232577 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ce590557-fbd0-4015-b858-b02ae4229aaf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:14:37.234121 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1439/2859] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:14:37 2022] DELETE /identity/v3/projects/963a3d7d5c0143a299527f020e7a37a6 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:14:37.239073 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f6d58-291e-4011-b796-87ca071598c5 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:14:37.239729 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f6d58-291e-4011-b796-87ca071598c5 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:14:37.239729 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56f6d58-291e-4011-b796-87ca071598c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:14:37.258509 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-a56f6d58-291e-4011-b796-87ca071598c5 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:14:37.259781 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-a56f6d58-291e-4011-b796-87ca071598c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:14:37.286273 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:14:37.319250 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1421/2860] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:14:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:14:37.327629 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85d901c-f459-4254-9f06-c74af93061b4 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:14:37.328022 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85d901c-f459-4254-9f06-c74af93061b4 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:14:37.328311 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85d901c-f459-4254-9f06-c74af93061b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:14:37.347052 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-f85d901c-f459-4254-9f06-c74af93061b4 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:14:37.348290 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-f85d901c-f459-4254-9f06-c74af93061b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:14:37.356089 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1440/2861] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:14:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:14:37.360170 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56f6d58-291e-4011-b796-87ca071598c5 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:14:37.369487 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56f6d58-291e-4011-b796-87ca071598c5 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:14:37.374008 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f272997-c4a7-4b59-a516-2bf05c2017c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:14:37.375961 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f272997-c4a7-4b59-a516-2bf05c2017c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:14:37.375961 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f272997-c4a7-4b59-a516-2bf05c2017c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:14:37.375961 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f272997-c4a7-4b59-a516-2bf05c2017c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:14:37.375961 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f272997-c4a7-4b59-a516-2bf05c2017c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80238) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 13:14:37.376332 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f272997-c4a7-4b59-a516-2bf05c2017c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:14:37.377800 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f272997-c4a7-4b59-a516-2bf05c2017c2 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:14:37.389666 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1422/2862] 10.222.0.27 () {58 vars in 1152 bytes} [Fri Jun 10 13:14:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 13:14:37.393749 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85d901c-f459-4254-9f06-c74af93061b4 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:14:37.403408 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85d901c-f459-4254-9f06-c74af93061b4 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:14:37.408413 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2db0c1-dea6-4766-a576-e234c56296cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:14:37.409053 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2db0c1-dea6-4766-a576-e234c56296cf admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:14:37.409340 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2db0c1-dea6-4766-a576-e234c56296cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:14:37.409620 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2db0c1-dea6-4766-a576-e234c56296cf admin admin] PATH_INFO: `/v3/projects` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:14:37.410298 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2db0c1-dea6-4766-a576-e234c56296cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:14:37.412192 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2db0c1-dea6-4766-a576-e234c56296cf admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:14:37.473061 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1441/2863] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:14:37 2022] POST /identity/v3/projects => generated 383 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:14:37.477057 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f272997-c4a7-4b59-a516-2bf05c2017c2 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:14:37.487264 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f272997-c4a7-4b59-a516-2bf05c2017c2 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:14:37.542594 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7d05d7-3249-4f93-aed0-2fc47c4c04a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:14:37.543255 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7d05d7-3249-4f93-aed0-2fc47c4c04a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:14:37.543533 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7d05d7-3249-4f93-aed0-2fc47c4c04a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:14:37.543804 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7d05d7-3249-4f93-aed0-2fc47c4c04a8 admin admin] PATH_INFO: `/v3/users` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:14:37.544519 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7d05d7-3249-4f93-aed0-2fc47c4c04a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:14:37.546408 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7d05d7-3249-4f93-aed0-2fc47c4c04a8 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:14:37.603911 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1423/2864] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:14:37 2022] POST /identity/v3/users => generated 292 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:14:37.608064 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2db0c1-dea6-4766-a576-e234c56296cf admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:14:37.618926 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2db0c1-dea6-4766-a576-e234c56296cf admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:14:37.623573 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c9fd8f-5982-4c17-b012-bb073b281734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:14:37.624263 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c9fd8f-5982-4c17-b012-bb073b281734 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:14:37.624549 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c9fd8f-5982-4c17-b012-bb073b281734 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:14:37.624826 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c9fd8f-5982-4c17-b012-bb073b281734 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:14:37.625572 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c9fd8f-5982-4c17-b012-bb073b281734 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:14:37.627251 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c9fd8f-5982-4c17-b012-bb073b281734 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:14:37.638381 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1442/2865] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:14:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:14:37.642315 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7d05d7-3249-4f93-aed0-2fc47c4c04a8 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:14:37.652308 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7d05d7-3249-4f93-aed0-2fc47c4c04a8 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:14:37.657727 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89735c4-12c7-4aa6-ad2f-464497ecf5b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:14:37.658375 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89735c4-12c7-4aa6-ad2f-464497ecf5b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:14:37.658660 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89735c4-12c7-4aa6-ad2f-464497ecf5b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:14:37.658940 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89735c4-12c7-4aa6-ad2f-464497ecf5b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:14:37.659658 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89735c4-12c7-4aa6-ad2f-464497ecf5b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:14:37.661268 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89735c4-12c7-4aa6-ad2f-464497ecf5b8 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:14:37.671329 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1424/2866] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:14:37 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:14:37.675493 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c9fd8f-5982-4c17-b012-bb073b281734 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:14:37.686198 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c9fd8f-5982-4c17-b012-bb073b281734 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:14:37.691143 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469f5ef7-c077-49c6-b8ec-0eaa8f9dbf0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:14:37.692123 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469f5ef7-c077-49c6-b8ec-0eaa8f9dbf0c admin admin] REQUEST_METHOD: `PUT` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:14:37.692435 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469f5ef7-c077-49c6-b8ec-0eaa8f9dbf0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:14:37.692711 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469f5ef7-c077-49c6-b8ec-0eaa8f9dbf0c admin admin] PATH_INFO: `/v3/projects/c560fa0f4a7a4202bd8515936b9b0442/users/0d8caaacc31541209a633c54e3a41776/roles/43106c664e1b40de984c0c62701149c8` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:14:37.708546 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469f5ef7-c077-49c6-b8ec-0eaa8f9dbf0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c560fa0f4a7a4202bd8515936b9b0442, user_id=0d8caaacc31541209a633c54e3a41776, role_id=43106c664e1b40de984c0c62701149c8)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:14:37.711363 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469f5ef7-c077-49c6-b8ec-0eaa8f9dbf0c admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:14:37.757734 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1443/2867] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:14:37 2022] PUT /identity/v3/projects/c560fa0f4a7a4202bd8515936b9b0442/users/0d8caaacc31541209a633c54e3a41776/roles/43106c664e1b40de984c0c62701149c8 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:14:37.769451 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb19f47-c398-4726-82f7-13b8f6c04ffb None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:14:37.769849 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb19f47-c398-4726-82f7-13b8f6c04ffb None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:14:37.770230 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb19f47-c398-4726-82f7-13b8f6c04ffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:14:37.778055 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469f5ef7-c077-49c6-b8ec-0eaa8f9dbf0c admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:14:37.780753 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-469f5ef7-c077-49c6-b8ec-0eaa8f9dbf0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:14:37.799715 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-6eb19f47-c398-4726-82f7-13b8f6c04ffb None None] MFA Rules not processed for user `0d8caaacc31541209a633c54e3a41776`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:14:37.801066 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-6eb19f47-c398-4726-82f7-13b8f6c04ffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:14:37.859846 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469f5ef7-c077-49c6-b8ec-0eaa8f9dbf0c admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:14:37.873316 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13d7818-0422-47f7-8fbd-ec39983cb803 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bd6caa461e84100a3d5fe5af5d80f3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:14:37.874158 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13d7818-0422-47f7-8fbd-ec39983cb803 service nova] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:14:37.874498 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13d7818-0422-47f7-8fbd-ec39983cb803 service nova] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:14:37.874918 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13d7818-0422-47f7-8fbd-ec39983cb803 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:14:37.876753 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a13d7818-0422-47f7-8fbd-ec39983cb803 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:14:37.888059 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1425/2868] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:14:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:14:37.891482 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13d7818-0422-47f7-8fbd-ec39983cb803 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:14:37.892512 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb19f47-c398-4726-82f7-13b8f6c04ffb None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:14:37.893738 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13d7818-0422-47f7-8fbd-ec39983cb803 service nova] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:14:37.894278 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-6eb19f47-c398-4726-82f7-13b8f6c04ffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:14:37.906101 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1444/2869] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:14:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 10 13:14:37.927728 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:14:37.954945 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb19f47-c398-4726-82f7-13b8f6c04ffb None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:14:37.958655 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b54e6b-a29e-4e51-9955-ca9fd1f21713 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6cc95a563b941ac92ab283aa0d5a5d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:14:37.959291 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b54e6b-a29e-4e51-9955-ca9fd1f21713 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:14:37.959578 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b54e6b-a29e-4e51-9955-ca9fd1f21713 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:14:37.959853 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b54e6b-a29e-4e51-9955-ca9fd1f21713 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:14:37.968253 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b54e6b-a29e-4e51-9955-ca9fd1f21713 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:14:37.969771 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b54e6b-a29e-4e51-9955-ca9fd1f21713 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:14:37.979437 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1426/2870] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:14:37 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 10 13:14:40.957337 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13d7818-0422-47f7-8fbd-ec39983cb803 service nova] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:14:40.991379 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=91,keystone:DELETE=6,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:14:41.019535 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13d7818-0422-47f7-8fbd-ec39983cb803 service nova] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:14:41.024384 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16f485a-90ae-4ab3-a3e4-8c04e3a084b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:14:41.025281 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16f485a-90ae-4ab3-a3e4-8c04e3a084b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:14:41.025595 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16f485a-90ae-4ab3-a3e4-8c04e3a084b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:14:41.025871 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16f485a-90ae-4ab3-a3e4-8c04e3a084b2 admin admin] PATH_INFO: `/v3/users/0d8caaacc31541209a633c54e3a41776` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:14:41.027078 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16f485a-90ae-4ab3-a3e4-8c04e3a084b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d8caaacc31541209a633c54e3a41776)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:14:41.029067 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16f485a-90ae-4ab3-a3e4-8c04e3a084b2 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:14:41.200079 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d16f485a-90ae-4ab3-a3e4-8c04e3a084b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d8caaacc31541209a633c54e3a41776'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:14:41.206979 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d16f485a-90ae-4ab3-a3e4-8c04e3a084b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d8caaacc31541209a633c54e3a41776'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:14:41.254657 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d16f485a-90ae-4ab3-a3e4-8c04e3a084b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d8caaacc31541209a633c54e3a41776'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:14:41.255278 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d16f485a-90ae-4ab3-a3e4-8c04e3a084b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d8caaacc31541209a633c54e3a41776'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:14:41.267994 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-d16f485a-90ae-4ab3-a3e4-8c04e3a084b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d8caaacc31541209a633c54e3a41776'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:14:41.273926 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1445/2871] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:14:40 2022] DELETE /identity/v3/users/0d8caaacc31541209a633c54e3a41776 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:14:41.278659 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cecfcdb-e0bb-43fd-924d-9480028f71eb None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:14:41.279126 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cecfcdb-e0bb-43fd-924d-9480028f71eb None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:14:41.279593 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cecfcdb-e0bb-43fd-924d-9480028f71eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:14:41.297249 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-2cecfcdb-e0bb-43fd-924d-9480028f71eb None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:14:41.298439 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-2cecfcdb-e0bb-43fd-924d-9480028f71eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:14:41.359347 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1427/2872] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:14:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 10 13:14:41.366019 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16f485a-90ae-4ab3-a3e4-8c04e3a084b2 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:14:41.367397 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-d16f485a-90ae-4ab3-a3e4-8c04e3a084b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:14:41.426902 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16f485a-90ae-4ab3-a3e4-8c04e3a084b2 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:14:41.430570 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f12ae9e-7052-45c9-9c57-3f65fce53f27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:14:41.431189 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f12ae9e-7052-45c9-9c57-3f65fce53f27 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:14:41.431476 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f12ae9e-7052-45c9-9c57-3f65fce53f27 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:14:41.431756 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f12ae9e-7052-45c9-9c57-3f65fce53f27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:14:41.448858 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f12ae9e-7052-45c9-9c57-3f65fce53f27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:14:41.450482 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f12ae9e-7052-45c9-9c57-3f65fce53f27 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:14:41.461259 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1446/2873] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:14:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:14:41.839474 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cecfcdb-e0bb-43fd-924d-9480028f71eb None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:14:41.840984 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-2cecfcdb-e0bb-43fd-924d-9480028f71eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:14:41.853442 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cecfcdb-e0bb-43fd-924d-9480028f71eb None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:14:41.859564 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f778836c-697b-4845-9529-9ca7d8f72072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:14:41.860501 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f778836c-697b-4845-9529-9ca7d8f72072 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:14:41.860804 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f778836c-697b-4845-9529-9ca7d8f72072 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:14:41.861124 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f778836c-697b-4845-9529-9ca7d8f72072 admin admin] PATH_INFO: `/v3/projects/c560fa0f4a7a4202bd8515936b9b0442` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:14:41.862277 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f778836c-697b-4845-9529-9ca7d8f72072 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c560fa0f4a7a4202bd8515936b9b0442)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:14:41.874987 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f778836c-697b-4845-9529-9ca7d8f72072 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:14:41.999089 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f778836c-697b-4845-9529-9ca7d8f72072 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c560fa0f4a7a4202bd8515936b9b0442', 'request_id': 'req-f778836c-697b-4845-9529-9ca7d8f72072'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:14:42.004472 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f778836c-697b-4845-9529-9ca7d8f72072 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c560fa0f4a7a4202bd8515936b9b0442', 'request_id': 'req-f778836c-697b-4845-9529-9ca7d8f72072'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:14:42.010863 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f778836c-697b-4845-9529-9ca7d8f72072 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c560fa0f4a7a4202bd8515936b9b0442', 'request_id': 'req-f778836c-697b-4845-9529-9ca7d8f72072'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:14:42.105258 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f778836c-697b-4845-9529-9ca7d8f72072 admin admin] The token cache is being invalidate because project c560fa0f4a7a4202bd8515936b9b0442 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:14:42.106049 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f778836c-697b-4845-9529-9ca7d8f72072 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:14:42.107011 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-f778836c-697b-4845-9529-9ca7d8f72072 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:14:42.108741 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1428/2874] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:14:41 2022] DELETE /identity/v3/projects/c560fa0f4a7a4202bd8515936b9b0442 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:14:42.113732 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdc827c-fda4-4706-8111-6c5aad92ea04 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:14:42.114181 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdc827c-fda4-4706-8111-6c5aad92ea04 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:14:42.114277 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdc827c-fda4-4706-8111-6c5aad92ea04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:14:42.132079 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-8bdc827c-fda4-4706-8111-6c5aad92ea04 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:14:42.133297 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8bdc827c-fda4-4706-8111-6c5aad92ea04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:14:42.180167 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:14:42.191370 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1447/2875] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:14:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 10 13:14:42.199310 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2b7ffb-7964-4c37-ba26-28d057481e23 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:14:42.199602 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2b7ffb-7964-4c37-ba26-28d057481e23 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:14:42.199914 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2b7ffb-7964-4c37-ba26-28d057481e23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:14:42.218473 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-7a2b7ffb-7964-4c37-ba26-28d057481e23 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:14:42.219669 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-7a2b7ffb-7964-4c37-ba26-28d057481e23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:14:42.227552 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1429/2876] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:14:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:14:42.232580 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdc827c-fda4-4706-8111-6c5aad92ea04 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:14:42.242299 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdc827c-fda4-4706-8111-6c5aad92ea04 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:14:42.246897 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa601de-4987-4617-af6c-afef1c7c9a8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:14:42.247550 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa601de-4987-4617-af6c-afef1c7c9a8a admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:14:42.247916 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa601de-4987-4617-af6c-afef1c7c9a8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:14:42.248396 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa601de-4987-4617-af6c-afef1c7c9a8a admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:14:42.249343 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa601de-4987-4617-af6c-afef1c7c9a8a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 13:14:42.249980 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa601de-4987-4617-af6c-afef1c7c9a8a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:14:42.251611 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa601de-4987-4617-af6c-afef1c7c9a8a admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:14:42.263707 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1448/2877] 10.222.0.27 () {58 vars in 1152 bytes} [Fri Jun 10 13:14:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 10 13:14:42.267823 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2b7ffb-7964-4c37-ba26-28d057481e23 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:14:42.276911 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2b7ffb-7964-4c37-ba26-28d057481e23 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:14:42.282255 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef19231d-0367-4e05-8c10-ed1dab83df74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:14:42.282898 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef19231d-0367-4e05-8c10-ed1dab83df74 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:14:42.283180 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef19231d-0367-4e05-8c10-ed1dab83df74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:14:42.283453 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef19231d-0367-4e05-8c10-ed1dab83df74 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:14:42.284164 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef19231d-0367-4e05-8c10-ed1dab83df74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:14:42.286048 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef19231d-0367-4e05-8c10-ed1dab83df74 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:14:42.347248 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1430/2878] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:14:42 2022] POST /identity/v3/projects => generated 389 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:14:42.351304 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa601de-4987-4617-af6c-afef1c7c9a8a admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:14:42.361739 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa601de-4987-4617-af6c-afef1c7c9a8a admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:14:42.421931 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4e5840-6226-4c2f-8174-ebc4caca05c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:14:42.422591 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4e5840-6226-4c2f-8174-ebc4caca05c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:14:42.422876 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4e5840-6226-4c2f-8174-ebc4caca05c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:14:42.423152 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4e5840-6226-4c2f-8174-ebc4caca05c1 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:14:42.423827 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4e5840-6226-4c2f-8174-ebc4caca05c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:14:42.429561 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4e5840-6226-4c2f-8174-ebc4caca05c1 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:14:42.523899 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1449/2879] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:14:42 2022] POST /identity/v3/users => generated 295 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:14:42.529716 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef19231d-0367-4e05-8c10-ed1dab83df74 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:14:42.545951 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef19231d-0367-4e05-8c10-ed1dab83df74 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:14:42.552211 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53bc949-3a95-4a66-8cc5-915c559ad1c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:14:42.553608 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53bc949-3a95-4a66-8cc5-915c559ad1c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:14:42.554128 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53bc949-3a95-4a66-8cc5-915c559ad1c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:14:42.554411 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53bc949-3a95-4a66-8cc5-915c559ad1c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:14:42.555216 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53bc949-3a95-4a66-8cc5-915c559ad1c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:14:42.556882 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53bc949-3a95-4a66-8cc5-915c559ad1c9 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:14:42.568832 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1431/2880] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:14:42 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:14:42.572966 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4e5840-6226-4c2f-8174-ebc4caca05c1 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:14:42.583449 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4e5840-6226-4c2f-8174-ebc4caca05c1 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:14:42.587981 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41422112-16c6-4629-b681-7dc4e01107e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:14:42.588632 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41422112-16c6-4629-b681-7dc4e01107e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:14:42.588914 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41422112-16c6-4629-b681-7dc4e01107e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:14:42.589189 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41422112-16c6-4629-b681-7dc4e01107e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:14:42.589906 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41422112-16c6-4629-b681-7dc4e01107e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:14:42.592145 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41422112-16c6-4629-b681-7dc4e01107e2 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:14:42.601819 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1450/2881] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:14:42 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 10 13:14:42.605738 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53bc949-3a95-4a66-8cc5-915c559ad1c9 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:14:42.615502 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53bc949-3a95-4a66-8cc5-915c559ad1c9 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:14:42.620088 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc85f9d9-e3bc-4729-b099-1de36ad889b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:14:42.620957 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc85f9d9-e3bc-4729-b099-1de36ad889b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:14:42.621257 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc85f9d9-e3bc-4729-b099-1de36ad889b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:14:42.621515 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc85f9d9-e3bc-4729-b099-1de36ad889b5 admin admin] PATH_INFO: `/v3/projects/e503306725964bc1b138654161a59891/users/870c0a7b962e4428b3869ab068d58df4/roles/43106c664e1b40de984c0c62701149c8` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:14:42.633326 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:14:42.637196 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc85f9d9-e3bc-4729-b099-1de36ad889b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e503306725964bc1b138654161a59891, user_id=870c0a7b962e4428b3869ab068d58df4, role_id=43106c664e1b40de984c0c62701149c8)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:14:42.640407 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc85f9d9-e3bc-4729-b099-1de36ad889b5 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:14:42.717931 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1432/2882] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:14:42 2022] PUT /identity/v3/projects/e503306725964bc1b138654161a59891/users/870c0a7b962e4428b3869ab068d58df4/roles/43106c664e1b40de984c0c62701149c8 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:14:42.728181 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88540659-0980-4bf5-aa4f-1dd12297d565 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:14:42.728472 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88540659-0980-4bf5-aa4f-1dd12297d565 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:14:42.728753 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88540659-0980-4bf5-aa4f-1dd12297d565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:14:42.754376 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-88540659-0980-4bf5-aa4f-1dd12297d565 None None] MFA Rules not processed for user `870c0a7b962e4428b3869ab068d58df4`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:14:42.755514 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-88540659-0980-4bf5-aa4f-1dd12297d565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:14:42.818327 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1451/2883] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:14:42 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:14:42.829372 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc85f9d9-e3bc-4729-b099-1de36ad889b5 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:14:42.831039 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-dc85f9d9-e3bc-4729-b099-1de36ad889b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:14:42.889038 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc85f9d9-e3bc-4729-b099-1de36ad889b5 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:14:42.892845 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e6fdb9-7153-48ee-825a-8d11e7f12cb2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6cc95a563b941ac92ab283aa0d5a5d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:14:42.893487 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e6fdb9-7153-48ee-825a-8d11e7f12cb2 service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:14:42.893852 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e6fdb9-7153-48ee-825a-8d11e7f12cb2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:14:42.894131 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e6fdb9-7153-48ee-825a-8d11e7f12cb2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:14:42.902735 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e6fdb9-7153-48ee-825a-8d11e7f12cb2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:14:42.904453 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e6fdb9-7153-48ee-825a-8d11e7f12cb2 service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:14:42.914438 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1433/2884] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:14:42 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:14:44.529375 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c505dc-9859-434e-98bd-2cdbb42985db None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:14:44.529826 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c505dc-9859-434e-98bd-2cdbb42985db None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:14:44.529956 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c505dc-9859-434e-98bd-2cdbb42985db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:14:44.547790 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-86c505dc-9859-434e-98bd-2cdbb42985db None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:14:44.549027 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-86c505dc-9859-434e-98bd-2cdbb42985db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:14:44.593756 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:14:44.607557 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1452/2885] 10.222.0.36 () {58 vars in 1016 bytes} [Fri Jun 10 13:14:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:14:44.634216 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e6fdb9-7153-48ee-825a-8d11e7f12cb2 service cinder] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:14:44.635739 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-53e6fdb9-7153-48ee-825a-8d11e7f12cb2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:14:44.714753 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e6fdb9-7153-48ee-825a-8d11e7f12cb2 service cinder] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:14:44.720220 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648832ee-d872-4814-8684-9c39686ef381 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bd6caa461e84100a3d5fe5af5d80f3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:14:44.720859 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648832ee-d872-4814-8684-9c39686ef381 service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:14:44.721139 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648832ee-d872-4814-8684-9c39686ef381 service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:14:44.721421 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648832ee-d872-4814-8684-9c39686ef381 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:14:44.729638 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648832ee-d872-4814-8684-9c39686ef381 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:14:44.731198 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648832ee-d872-4814-8684-9c39686ef381 service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:14:44.741397 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1434/2886] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:14:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:14:49.064978 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b47d3be-c752-49c9-ae8a-7b0487c53193 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:14:49.065458 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b47d3be-c752-49c9-ae8a-7b0487c53193 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:14:49.065458 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b47d3be-c752-49c9-ae8a-7b0487c53193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:14:49.087343 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-3b47d3be-c752-49c9-ae8a-7b0487c53193 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:14:49.087343 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-3b47d3be-c752-49c9-ae8a-7b0487c53193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:14:49.093435 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1453/2887] 10.222.0.36 () {58 vars in 1016 bytes} [Fri Jun 10 13:14:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:14:49.113384 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648832ee-d872-4814-8684-9c39686ef381 service nova] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:14:49.115621 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-648832ee-d872-4814-8684-9c39686ef381 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:14:49.130927 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648832ee-d872-4814-8684-9c39686ef381 service nova] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:14:49.140351 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb661910-fa46-4f6d-9b45-875d09f0699e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bd6caa461e84100a3d5fe5af5d80f3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:14:49.141009 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb661910-fa46-4f6d-9b45-875d09f0699e service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:14:49.141296 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb661910-fa46-4f6d-9b45-875d09f0699e service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:14:49.141580 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb661910-fa46-4f6d-9b45-875d09f0699e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:14:49.150638 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb661910-fa46-4f6d-9b45-875d09f0699e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:14:49.152271 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb661910-fa46-4f6d-9b45-875d09f0699e service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:14:49.162534 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1435/2888] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:14:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:14:51.799798 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843d94cc-abd6-42ae-b60b-ba8c6a863edf None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:14:51.800294 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843d94cc-abd6-42ae-b60b-ba8c6a863edf None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:14:51.800414 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843d94cc-abd6-42ae-b60b-ba8c6a863edf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:14:51.818324 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-843d94cc-abd6-42ae-b60b-ba8c6a863edf None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:14:51.819511 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-843d94cc-abd6-42ae-b60b-ba8c6a863edf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:14:51.827677 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1454/2889] 10.222.0.36 () {58 vars in 1016 bytes} [Fri Jun 10 13:14:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:14:51.851275 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb661910-fa46-4f6d-9b45-875d09f0699e service nova] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:14:51.863970 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb661910-fa46-4f6d-9b45-875d09f0699e service nova] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:14:51.871231 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fe7f97-c2fb-45d3-b4b9-920fa8810a87 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bd6caa461e84100a3d5fe5af5d80f3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:14:51.871231 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fe7f97-c2fb-45d3-b4b9-920fa8810a87 service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:14:51.871564 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fe7f97-c2fb-45d3-b4b9-920fa8810a87 service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:14:51.871685 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fe7f97-c2fb-45d3-b4b9-920fa8810a87 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:14:51.884340 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fe7f97-c2fb-45d3-b4b9-920fa8810a87 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:14:51.886605 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fe7f97-c2fb-45d3-b4b9-920fa8810a87 service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:14:51.897594 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1436/2890] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:14:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:14:54.371485 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843d94cc-abd6-42ae-b60b-ba8c6a863edf None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:14:54.381716 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843d94cc-abd6-42ae-b60b-ba8c6a863edf None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:14:54.386200 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de6b23a-bc5f-4569-b21a-45f81b2380bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:14:54.387069 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de6b23a-bc5f-4569-b21a-45f81b2380bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:14:54.387356 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de6b23a-bc5f-4569-b21a-45f81b2380bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:14:54.387658 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de6b23a-bc5f-4569-b21a-45f81b2380bb admin admin] PATH_INFO: `/v3/users/870c0a7b962e4428b3869ab068d58df4` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:14:54.389518 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de6b23a-bc5f-4569-b21a-45f81b2380bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=870c0a7b962e4428b3869ab068d58df4)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:14:54.391407 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de6b23a-bc5f-4569-b21a-45f81b2380bb admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:14:54.587401 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0de6b23a-bc5f-4569-b21a-45f81b2380bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '870c0a7b962e4428b3869ab068d58df4'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:14:54.594043 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0de6b23a-bc5f-4569-b21a-45f81b2380bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '870c0a7b962e4428b3869ab068d58df4'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:14:54.660136 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0de6b23a-bc5f-4569-b21a-45f81b2380bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '870c0a7b962e4428b3869ab068d58df4'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:14:54.660822 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0de6b23a-bc5f-4569-b21a-45f81b2380bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '870c0a7b962e4428b3869ab068d58df4'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:14:54.673122 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-0de6b23a-bc5f-4569-b21a-45f81b2380bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '870c0a7b962e4428b3869ab068d58df4'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:14:54.678903 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1455/2891] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:14:54 2022] DELETE /identity/v3/users/870c0a7b962e4428b3869ab068d58df4 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:14:54.683376 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21724110-aabc-48ca-8afc-3dddd7515d35 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:14:54.701311 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21724110-aabc-48ca-8afc-3dddd7515d35 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:14:54.701311 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21724110-aabc-48ca-8afc-3dddd7515d35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:14:54.701615 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-21724110-aabc-48ca-8afc-3dddd7515d35 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:14:54.702804 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-21724110-aabc-48ca-8afc-3dddd7515d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:14:54.745821 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:14:54.759985 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1437/2892] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:14:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:14:54.769796 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de6b23a-bc5f-4569-b21a-45f81b2380bb admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:14:54.771173 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-0de6b23a-bc5f-4569-b21a-45f81b2380bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:14:54.828497 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de6b23a-bc5f-4569-b21a-45f81b2380bb admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:14:54.832187 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339af440-5484-4ac3-bbad-fd8802073c1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:14:54.832942 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339af440-5484-4ac3-bbad-fd8802073c1e service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:14:54.833236 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339af440-5484-4ac3-bbad-fd8802073c1e service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:14:54.833519 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339af440-5484-4ac3-bbad-fd8802073c1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:14:54.843521 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339af440-5484-4ac3-bbad-fd8802073c1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:14:54.844986 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339af440-5484-4ac3-bbad-fd8802073c1e service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:14:54.858093 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1456/2893] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:14:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 10 13:14:55.222195 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21724110-aabc-48ca-8afc-3dddd7515d35 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:14:55.223633 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-21724110-aabc-48ca-8afc-3dddd7515d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:14:55.236644 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21724110-aabc-48ca-8afc-3dddd7515d35 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:14:55.242226 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852c7986-6ba0-4d8a-a95d-a955a44f9165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:14:55.243249 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852c7986-6ba0-4d8a-a95d-a955a44f9165 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:14:55.243538 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852c7986-6ba0-4d8a-a95d-a955a44f9165 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:14:55.243810 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852c7986-6ba0-4d8a-a95d-a955a44f9165 admin admin] PATH_INFO: `/v3/projects/e503306725964bc1b138654161a59891` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:14:55.245030 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852c7986-6ba0-4d8a-a95d-a955a44f9165 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e503306725964bc1b138654161a59891)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:14:55.247128 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852c7986-6ba0-4d8a-a95d-a955a44f9165 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:14:55.351553 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-852c7986-6ba0-4d8a-a95d-a955a44f9165 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e503306725964bc1b138654161a59891', 'request_id': 'req-852c7986-6ba0-4d8a-a95d-a955a44f9165'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:14:55.356884 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-852c7986-6ba0-4d8a-a95d-a955a44f9165 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e503306725964bc1b138654161a59891', 'request_id': 'req-852c7986-6ba0-4d8a-a95d-a955a44f9165'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:14:55.363827 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-852c7986-6ba0-4d8a-a95d-a955a44f9165 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e503306725964bc1b138654161a59891', 'request_id': 'req-852c7986-6ba0-4d8a-a95d-a955a44f9165'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:14:55.430738 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-852c7986-6ba0-4d8a-a95d-a955a44f9165 admin admin] The token cache is being invalidate because project e503306725964bc1b138654161a59891 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:14:55.431051 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-852c7986-6ba0-4d8a-a95d-a955a44f9165 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:14:55.431652 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-852c7986-6ba0-4d8a-a95d-a955a44f9165 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:14:55.433240 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1438/2894] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:14:55 2022] DELETE /identity/v3/projects/e503306725964bc1b138654161a59891 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:14:56.056776 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a06674-6df8-4ac0-8ae7-7de971e12343 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:14:56.057271 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a06674-6df8-4ac0-8ae7-7de971e12343 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:14:56.057381 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a06674-6df8-4ac0-8ae7-7de971e12343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:14:56.074319 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-a3a06674-6df8-4ac0-8ae7-7de971e12343 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:14:56.076021 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a3a06674-6df8-4ac0-8ae7-7de971e12343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:14:56.122287 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:14:56.134417 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1457/2895] 10.222.0.36 () {58 vars in 1016 bytes} [Fri Jun 10 13:14:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 10 13:14:56.157688 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852c7986-6ba0-4d8a-a95d-a955a44f9165 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:14:56.159049 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-852c7986-6ba0-4d8a-a95d-a955a44f9165 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:14:56.235258 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852c7986-6ba0-4d8a-a95d-a955a44f9165 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:14:56.240678 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ed25df-b515-4da2-a2d1-b232ec950158 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bd6caa461e84100a3d5fe5af5d80f3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:14:56.241371 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ed25df-b515-4da2-a2d1-b232ec950158 service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:14:56.241663 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ed25df-b515-4da2-a2d1-b232ec950158 service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:14:56.241942 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ed25df-b515-4da2-a2d1-b232ec950158 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:14:56.250105 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ed25df-b515-4da2-a2d1-b232ec950158 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:14:56.251682 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ed25df-b515-4da2-a2d1-b232ec950158 service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:14:56.261101 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1439/2896] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:14:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:15:00.296693 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aaff9f-3e2c-4e5f-98cf-c44bc05b6a6e None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:15:00.297231 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aaff9f-3e2c-4e5f-98cf-c44bc05b6a6e None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:15:00.297231 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aaff9f-3e2c-4e5f-98cf-c44bc05b6a6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:15:00.315006 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-47aaff9f-3e2c-4e5f-98cf-c44bc05b6a6e None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:15:00.316236 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-47aaff9f-3e2c-4e5f-98cf-c44bc05b6a6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:15:00.323938 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1458/2897] 10.222.0.36 () {58 vars in 1016 bytes} [Fri Jun 10 13:15:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:15:00.349773 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ed25df-b515-4da2-a2d1-b232ec950158 service nova] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:15:00.351275 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-45ed25df-b515-4da2-a2d1-b232ec950158 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:15:00.366481 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ed25df-b515-4da2-a2d1-b232ec950158 service nova] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:15:00.371998 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06357ec6-a867-469b-81d5-6e2dc18a944a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8bd6caa461e84100a3d5fe5af5d80f3e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service', 'member', 'ResellerAdmin', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:15:00.372574 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06357ec6-a867-469b-81d5-6e2dc18a944a service nova] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:15:00.372862 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06357ec6-a867-469b-81d5-6e2dc18a944a service nova] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:15:00.373141 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06357ec6-a867-469b-81d5-6e2dc18a944a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:15:00.381325 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06357ec6-a867-469b-81d5-6e2dc18a944a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:15:00.382891 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06357ec6-a867-469b-81d5-6e2dc18a944a service nova] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:15:00.392285 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1440/2898] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:15:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:15:10.309674 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=6 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:15:10.388093 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=71,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:15:33.980953 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aaff9f-3e2c-4e5f-98cf-c44bc05b6a6e None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:15:33.982427 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-47aaff9f-3e2c-4e5f-98cf-c44bc05b6a6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:15:34.059473 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aaff9f-3e2c-4e5f-98cf-c44bc05b6a6e None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:15:34.064118 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5c2424-646b-4e64-b30f-60d9977ccbb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:15:34.064976 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5c2424-646b-4e64-b30f-60d9977ccbb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:15:34.065260 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5c2424-646b-4e64-b30f-60d9977ccbb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:15:34.065538 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5c2424-646b-4e64-b30f-60d9977ccbb1 admin admin] PATH_INFO: `/v3/users/7db0e455e2c841b5ae1e9b60f3dd0906` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:15:34.066724 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5c2424-646b-4e64-b30f-60d9977ccbb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7db0e455e2c841b5ae1e9b60f3dd0906)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:15:34.068644 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5c2424-646b-4e64-b30f-60d9977ccbb1 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:15:34.290502 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ef5c2424-646b-4e64-b30f-60d9977ccbb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7db0e455e2c841b5ae1e9b60f3dd0906'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:15:34.297161 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ef5c2424-646b-4e64-b30f-60d9977ccbb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7db0e455e2c841b5ae1e9b60f3dd0906'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:15:34.373850 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ef5c2424-646b-4e64-b30f-60d9977ccbb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7db0e455e2c841b5ae1e9b60f3dd0906'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:15:34.374460 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ef5c2424-646b-4e64-b30f-60d9977ccbb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7db0e455e2c841b5ae1e9b60f3dd0906'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:15:34.386445 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-ef5c2424-646b-4e64-b30f-60d9977ccbb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7db0e455e2c841b5ae1e9b60f3dd0906'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:15:34.392883 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1459/2899] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:15:33 2022] DELETE /identity/v3/users/7db0e455e2c841b5ae1e9b60f3dd0906 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:15:34.632813 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06357ec6-a867-469b-81d5-6e2dc18a944a service nova] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:15:34.680416 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06357ec6-a867-469b-81d5-6e2dc18a944a service nova] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:15:34.735293 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e5ee97-b205-482d-8310-ffb15e98ecc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:15:34.736196 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e5ee97-b205-482d-8310-ffb15e98ecc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:15:34.736479 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e5ee97-b205-482d-8310-ffb15e98ecc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:15:34.736760 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e5ee97-b205-482d-8310-ffb15e98ecc9 admin admin] PATH_INFO: `/v3/projects/dce47649142f4c60ae08a6f9f1732386` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:15:34.737939 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e5ee97-b205-482d-8310-ffb15e98ecc9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dce47649142f4c60ae08a6f9f1732386)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:15:34.739835 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e5ee97-b205-482d-8310-ffb15e98ecc9 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:15:34.853418 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-53e5ee97-b205-482d-8310-ffb15e98ecc9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dce47649142f4c60ae08a6f9f1732386', 'request_id': 'req-53e5ee97-b205-482d-8310-ffb15e98ecc9'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:15:34.859039 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-53e5ee97-b205-482d-8310-ffb15e98ecc9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dce47649142f4c60ae08a6f9f1732386', 'request_id': 'req-53e5ee97-b205-482d-8310-ffb15e98ecc9'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:15:34.865883 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-53e5ee97-b205-482d-8310-ffb15e98ecc9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dce47649142f4c60ae08a6f9f1732386', 'request_id': 'req-53e5ee97-b205-482d-8310-ffb15e98ecc9'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:15:34.950364 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-53e5ee97-b205-482d-8310-ffb15e98ecc9 admin admin] The token cache is being invalidate because project dce47649142f4c60ae08a6f9f1732386 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:15:34.950571 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-53e5ee97-b205-482d-8310-ffb15e98ecc9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:15:34.951167 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-53e5ee97-b205-482d-8310-ffb15e98ecc9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:15:34.952746 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1441/2900] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:15:34 2022] DELETE /identity/v3/projects/dce47649142f4c60ae08a6f9f1732386 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:15:34.957779 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e56a695-373a-4112-afb8-048f581c1405 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:15:34.962928 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e56a695-373a-4112-afb8-048f581c1405 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:15:34.962928 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e56a695-373a-4112-afb8-048f581c1405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:15:35.013828 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-7e56a695-373a-4112-afb8-048f581c1405 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:15:35.015031 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-7e56a695-373a-4112-afb8-048f581c1405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:15:35.047182 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:15:35.073149 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1460/2901] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:15:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:15:35.080860 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ecf618-7aad-4069-873d-6dd39e0d4969 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:15:35.083191 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ecf618-7aad-4069-873d-6dd39e0d4969 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:15:35.083191 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ecf618-7aad-4069-873d-6dd39e0d4969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:15:35.099556 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-94ecf618-7aad-4069-873d-6dd39e0d4969 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:15:35.100842 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-94ecf618-7aad-4069-873d-6dd39e0d4969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:15:35.108818 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1442/2902] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:15:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:15:35.113684 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e56a695-373a-4112-afb8-048f581c1405 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:15:35.123375 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e56a695-373a-4112-afb8-048f581c1405 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:15:35.129025 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a706bf-4e00-4c32-8160-ae0e90c36a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:15:35.129677 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a706bf-4e00-4c32-8160-ae0e90c36a4d admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:15:35.129957 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a706bf-4e00-4c32-8160-ae0e90c36a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:15:35.130233 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a706bf-4e00-4c32-8160-ae0e90c36a4d admin admin] PATH_INFO: `/v3/domains` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:15:35.131011 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a706bf-4e00-4c32-8160-ae0e90c36a4d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80237) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 10 13:15:35.131319 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a706bf-4e00-4c32-8160-ae0e90c36a4d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:15:35.132972 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a706bf-4e00-4c32-8160-ae0e90c36a4d admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:15:35.144388 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1461/2903] 10.222.0.27 () {58 vars in 1152 bytes} [Fri Jun 10 13:15:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 10 13:15:35.148632 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ecf618-7aad-4069-873d-6dd39e0d4969 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:15:35.158561 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ecf618-7aad-4069-873d-6dd39e0d4969 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:15:35.163717 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14da8332-25b8-42f6-88f8-e7139e093678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:15:35.164399 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14da8332-25b8-42f6-88f8-e7139e093678 admin admin] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:15:35.164686 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14da8332-25b8-42f6-88f8-e7139e093678 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:15:35.164967 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14da8332-25b8-42f6-88f8-e7139e093678 admin admin] PATH_INFO: `/v3/projects` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:15:35.165659 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14da8332-25b8-42f6-88f8-e7139e093678 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:15:35.167656 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14da8332-25b8-42f6-88f8-e7139e093678 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:15:35.233894 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1443/2904] 10.222.0.27 () {60 vars in 1152 bytes} [Fri Jun 10 13:15:35 2022] POST /identity/v3/projects => generated 397 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:15:35.235770 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a706bf-4e00-4c32-8160-ae0e90c36a4d admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:15:35.246226 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a706bf-4e00-4c32-8160-ae0e90c36a4d admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:15:35.303410 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b66bfa-1693-4479-b118-7b739a0c1947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:15:35.304098 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b66bfa-1693-4479-b118-7b739a0c1947 admin admin] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:15:35.304387 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b66bfa-1693-4479-b118-7b739a0c1947 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:15:35.304659 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b66bfa-1693-4479-b118-7b739a0c1947 admin admin] PATH_INFO: `/v3/users` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:15:35.305328 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b66bfa-1693-4479-b118-7b739a0c1947 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:15:35.307204 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b66bfa-1693-4479-b118-7b739a0c1947 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:15:35.383880 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1462/2905] 10.222.0.27 () {60 vars in 1143 bytes} [Fri Jun 10 13:15:35 2022] POST /identity/v3/users => generated 299 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 10 13:15:35.387841 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14da8332-25b8-42f6-88f8-e7139e093678 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:15:35.398171 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14da8332-25b8-42f6-88f8-e7139e093678 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:15:35.402737 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b6abb6-af74-4655-9811-88e64585654a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:15:35.403390 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b6abb6-af74-4655-9811-88e64585654a admin admin] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:15:35.403675 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b6abb6-af74-4655-9811-88e64585654a admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:15:35.403988 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b6abb6-af74-4655-9811-88e64585654a admin admin] PATH_INFO: `/v3/roles` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:15:35.404710 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b6abb6-af74-4655-9811-88e64585654a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:15:35.406559 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b6abb6-af74-4655-9811-88e64585654a admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:15:35.417758 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1444/2906] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:15:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:15:35.425630 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b66bfa-1693-4479-b118-7b739a0c1947 admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:15:35.435494 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b66bfa-1693-4479-b118-7b739a0c1947 admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:15:35.440078 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7ecf99-1fa0-4ae7-9ca1-c3588ea359d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:15:35.440717 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7ecf99-1fa0-4ae7-9ca1-c3588ea359d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:15:35.441005 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7ecf99-1fa0-4ae7-9ca1-c3588ea359d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:15:35.441285 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7ecf99-1fa0-4ae7-9ca1-c3588ea359d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:15:35.442000 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7ecf99-1fa0-4ae7-9ca1-c3588ea359d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:15:35.443562 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7ecf99-1fa0-4ae7-9ca1-c3588ea359d8 admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:15:35.453403 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1463/2907] 10.222.0.27 () {58 vars in 1121 bytes} [Fri Jun 10 13:15:35 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 10 13:15:35.457344 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b6abb6-af74-4655-9811-88e64585654a admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:15:35.467464 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b6abb6-af74-4655-9811-88e64585654a admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:15:35.472894 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405e23b4-4ab5-4f6b-9cfa-fa583e15e340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:15:35.473770 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405e23b4-4ab5-4f6b-9cfa-fa583e15e340 admin admin] REQUEST_METHOD: `PUT` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:15:35.474147 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405e23b4-4ab5-4f6b-9cfa-fa583e15e340 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:15:35.474428 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405e23b4-4ab5-4f6b-9cfa-fa583e15e340 admin admin] PATH_INFO: `/v3/projects/9d92c9a8a30445ac97141c5bdc577f1b/users/329cafd0d4bc4135bea674b37aeef57f/roles/43106c664e1b40de984c0c62701149c8` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:15:35.490516 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405e23b4-4ab5-4f6b-9cfa-fa583e15e340 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d92c9a8a30445ac97141c5bdc577f1b, user_id=329cafd0d4bc4135bea674b37aeef57f, role_id=43106c664e1b40de984c0c62701149c8)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:15:35.493109 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405e23b4-4ab5-4f6b-9cfa-fa583e15e340 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:15:35.495256 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:15:35.590880 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1445/2908] 10.222.0.27 () {60 vars in 1482 bytes} [Fri Jun 10 13:15:35 2022] PUT /identity/v3/projects/9d92c9a8a30445ac97141c5bdc577f1b/users/329cafd0d4bc4135bea674b37aeef57f/roles/43106c664e1b40de984c0c62701149c8 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 10 13:15:35.601736 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca69f605-1086-4d10-91fe-22928d189090 None None] REQUEST_METHOD: `POST` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:15:35.602032 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca69f605-1086-4d10-91fe-22928d189090 None None] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:15:35.602342 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca69f605-1086-4d10-91fe-22928d189090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:15:35.627886 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.auth.core [None req-ca69f605-1086-4d10-91fe-22928d189090 None None] MFA Rules not processed for user `329cafd0d4bc4135bea674b37aeef57f`. Rule list: `[]` (Enabled: `True`). {{(pid=80237) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:15:35.629175 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-ca69f605-1086-4d10-91fe-22928d189090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:15:35.691023 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1464/2909] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:15:35 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:15:35.702188 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405e23b4-4ab5-4f6b-9cfa-fa583e15e340 admin admin] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:15:35.703854 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-405e23b4-4ab5-4f6b-9cfa-fa583e15e340 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:15:35.776010 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405e23b4-4ab5-4f6b-9cfa-fa583e15e340 admin admin] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:15:35.779383 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079b4d1d-1cd0-4370-bb70-3bb93c37f9ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c6cc95a563b941ac92ab283aa0d5a5d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:15:35.780041 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079b4d1d-1cd0-4370-bb70-3bb93c37f9ed service cinder] REQUEST_METHOD: `GET` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:15:35.780327 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079b4d1d-1cd0-4370-bb70-3bb93c37f9ed service cinder] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:15:35.780857 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079b4d1d-1cd0-4370-bb70-3bb93c37f9ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:15:35.789446 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079b4d1d-1cd0-4370-bb70-3bb93c37f9ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:15:35.790862 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079b4d1d-1cd0-4370-bb70-3bb93c37f9ed service cinder] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:15:35.799439 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1446/2910] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:15:35 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:15:41.663630 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca69f605-1086-4d10-91fe-22928d189090 None None] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:15:41.702809 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:15:41.726100 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca69f605-1086-4d10-91fe-22928d189090 None None] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:15:41.730644 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c902ae4-8fb3-4e5f-b23b-8a84b42397ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:15:41.731576 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c902ae4-8fb3-4e5f-b23b-8a84b42397ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:15:41.731880 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c902ae4-8fb3-4e5f-b23b-8a84b42397ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:15:41.732178 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c902ae4-8fb3-4e5f-b23b-8a84b42397ca admin admin] PATH_INFO: `/v3/users/329cafd0d4bc4135bea674b37aeef57f` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:15:41.733367 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c902ae4-8fb3-4e5f-b23b-8a84b42397ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=329cafd0d4bc4135bea674b37aeef57f)` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:15:41.735833 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c902ae4-8fb3-4e5f-b23b-8a84b42397ca admin admin] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:15:41.928963 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8c902ae4-8fb3-4e5f-b23b-8a84b42397ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '329cafd0d4bc4135bea674b37aeef57f'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:15:41.935967 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8c902ae4-8fb3-4e5f-b23b-8a84b42397ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '329cafd0d4bc4135bea674b37aeef57f'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:15:42.020096 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8c902ae4-8fb3-4e5f-b23b-8a84b42397ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '329cafd0d4bc4135bea674b37aeef57f'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:15:42.020725 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8c902ae4-8fb3-4e5f-b23b-8a84b42397ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '329cafd0d4bc4135bea674b37aeef57f'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:15:42.033396 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.notifications [None req-8c902ae4-8fb3-4e5f-b23b-8a84b42397ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '329cafd0d4bc4135bea674b37aeef57f'} {{(pid=80237) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:15:42.039475 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1465/2911] 10.222.0.27 () {58 vars in 1223 bytes} [Fri Jun 10 13:15:41 2022] DELETE /identity/v3/users/329cafd0d4bc4135bea674b37aeef57f => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:15:42.044066 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73340845-e33c-4db8-86fc-436ee1b9d0b9 None None] REQUEST_METHOD: `POST` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:15:42.044544 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73340845-e33c-4db8-86fc-436ee1b9d0b9 None None] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:15:42.044684 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73340845-e33c-4db8-86fc-436ee1b9d0b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:15:42.062135 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.auth.core [None req-73340845-e33c-4db8-86fc-436ee1b9d0b9 None None] MFA Rules not processed for user `85098428641c41389cc157b4c057067c`. Rule list: `[]` (Enabled: `True`). {{(pid=80238) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 10 13:15:42.063607 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-73340845-e33c-4db8-86fc-436ee1b9d0b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:15:42.120726 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1447/2912] 10.222.0.27 () {58 vars in 957 bytes} [Fri Jun 10 13:15:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 10 13:15:42.130618 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c902ae4-8fb3-4e5f-b23b-8a84b42397ca admin admin] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:15:42.132090 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-8c902ae4-8fb3-4e5f-b23b-8a84b42397ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:15:42.210017 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c902ae4-8fb3-4e5f-b23b-8a84b42397ca admin admin] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:15:42.213573 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaf465d-44fc-4b9d-95d1-a1988a1508e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc4415461a974c24ad7105a97c7d915b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:15:42.214211 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaf465d-44fc-4b9d-95d1-a1988a1508e2 service neutron] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:15:42.214494 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaf465d-44fc-4b9d-95d1-a1988a1508e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:15:42.214768 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaf465d-44fc-4b9d-95d1-a1988a1508e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:15:42.223671 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaf465d-44fc-4b9d-95d1-a1988a1508e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:15:42.225118 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbaf465d-44fc-4b9d-95d1-a1988a1508e2 service neutron] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:15:42.234365 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1466/2913] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:15:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:15:42.603241 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73340845-e33c-4db8-86fc-436ee1b9d0b9 None None] Authenticating user token {{(pid=80238) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:15:42.604732 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.fernet_utils [None req-73340845-e33c-4db8-86fc-436ee1b9d0b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80238) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:15:42.617898 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73340845-e33c-4db8-86fc-436ee1b9d0b9 None None] Validating token access rules against request {{(pid=80238) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:15:42.622602 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50283dc-139a-43fb-b352-12338260b968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '85098428641c41389cc157b4c057067c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e3a9af772214953842e9cc63a52ae3a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80238) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:15:42.623471 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50283dc-139a-43fb-b352-12338260b968 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:15:42.623752 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50283dc-139a-43fb-b352-12338260b968 admin admin] SCRIPT_NAME: `/identity` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:15:42.624062 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50283dc-139a-43fb-b352-12338260b968 admin admin] PATH_INFO: `/v3/projects/9d92c9a8a30445ac97141c5bdc577f1b` {{(pid=80238) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:15:42.625258 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50283dc-139a-43fb-b352-12338260b968 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d92c9a8a30445ac97141c5bdc577f1b)` {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:15:42.627182 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50283dc-139a-43fb-b352-12338260b968 admin admin] RBAC: Authorization granted {{(pid=80238) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:15:42.733239 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b50283dc-139a-43fb-b352-12338260b968 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d92c9a8a30445ac97141c5bdc577f1b', 'request_id': 'req-b50283dc-139a-43fb-b352-12338260b968'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:15:42.738713 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b50283dc-139a-43fb-b352-12338260b968 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d92c9a8a30445ac97141c5bdc577f1b', 'request_id': 'req-b50283dc-139a-43fb-b352-12338260b968'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:15:42.745382 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b50283dc-139a-43fb-b352-12338260b968 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d92c9a8a30445ac97141c5bdc577f1b', 'request_id': 'req-b50283dc-139a-43fb-b352-12338260b968'} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:15:42.824248 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b50283dc-139a-43fb-b352-12338260b968 admin admin] The token cache is being invalidate because project 9d92c9a8a30445ac97141c5bdc577f1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80238) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 10 13:15:42.824533 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b50283dc-139a-43fb-b352-12338260b968 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:15:42.825137 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG keystone.notifications [None req-b50283dc-139a-43fb-b352-12338260b968 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80238) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 10 13:15:42.826742 dv-cinder-master-3-smb devstack@keystone.service[80238]: [pid: 80238|app: 0|req: 1448/2914] 10.222.0.27 () {58 vars in 1232 bytes} [Fri Jun 10 13:15:42 2022] DELETE /identity/v3/projects/9d92c9a8a30445ac97141c5bdc577f1b => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 10 13:15:52.230213 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=72,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:15:52.821254 dv-cinder-master-3-smb devstack@keystone.service[80238]: DEBUG dbcounter [-] [80238] Writing DB stats keystone:INSERT=2,keystone:SELECT=89,keystone:DELETE=7 {{(pid=80238) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 10 13:17:22.311594 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaf465d-44fc-4b9d-95d1-a1988a1508e2 service neutron] Authenticating user token {{(pid=80237) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 10 13:17:22.313006 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-fbaf465d-44fc-4b9d-95d1-a1988a1508e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:17:22.400495 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbaf465d-44fc-4b9d-95d1-a1988a1508e2 service neutron] Validating token access rules against request {{(pid=80237) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 10 13:17:22.456780 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b395d0-c3c5-4032-847a-506daaa26895 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd972ab782b3f4d2aba575314f16dafc7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '73ea5b23a2d34544ab493f4c5e4034b8', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80237) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 10 13:17:22.457416 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b395d0-c3c5-4032-847a-506daaa26895 service placement] REQUEST_METHOD: `GET` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 10 13:17:22.457700 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b395d0-c3c5-4032-847a-506daaa26895 service placement] SCRIPT_NAME: `/identity` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 10 13:17:22.457974 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b395d0-c3c5-4032-847a-506daaa26895 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80237) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 10 13:17:22.459703 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.fernet_utils [None req-a4b395d0-c3c5-4032-847a-506daaa26895 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80237) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 10 13:17:22.470453 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b395d0-c3c5-4032-847a-506daaa26895 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 10 13:17:22.472022 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b395d0-c3c5-4032-847a-506daaa26895 service placement] RBAC: Authorization granted {{(pid=80237) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 10 13:17:22.482241 dv-cinder-master-3-smb devstack@keystone.service[80237]: [pid: 80237|app: 0|req: 1467/2915] 10.222.0.27 () {60 vars in 1365 bytes} [Fri Jun 10 13:17:22 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 10 13:17:32.477508 dv-cinder-master-3-smb devstack@keystone.service[80237]: DEBUG dbcounter [-] [80237] Writing DB stats keystone:SELECT=67 {{(pid=80237) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}